Annals of Mathematics, 159 (2004), 641-724 



Holomorphic extensions 

of representations: 
(I) automorphic functions 

By Bernhard Krotz and Robert J. Stanton* 
Abstract 

Let G be a connected, real, semisimple Lie group contained in its complex- 
ification Gc, and let K be a maximal compact subgroup of G. We construct 
a Kc~G double coset domain in Gc, and we show that the action of G on the 
A-finite vectors of any irreducible unitary representation of G has a holo- 
morphic extension to this domain. For the resultant holomorphic extension 
of A-finite matrix coefficients we obtain estimates of the singularities at the 
boundary, as well as majorant /minor ant estimates along the boundary. We 
obtain L°° bounds on holomorphically extended automorphic functions on 
G/K in terms of Sobolev norms, and we use these to estimate the Fourier 
coefficients of combinations of automorphic functions in a number of cases, 
e.g. of triple products of MaaB forms. 

Introduction 

Complex analysis played an important role in the classical development of 
the theory of Fourier series. However, even for S1(2,R) contained in S1(2,C), 
complex analysis on Sl(2, C) has had little impact on the harmonic analysis 
of S1(2,R). As the A-finite matrix coefficients of an irreducible unitary rep- 
resentation of S1(2,R) can be identified with classical special functions, such 
as hypergeometric functions, one knows they have holomorphic extensions to 
some domain. So for any infinite dimensional irreducible unitary representa- 
tion of S1(2,R), one can expect at most some proper subdomain of S1(2,C) to 
occur. It is less clear that there is a universal domain in Sl(2, C) to which the 
action of G on A-finite vectors of every irreducible unitary representation has 
holomorphic extension. One goal of this paper is to construct such a domain 
for a real, connected, semisimple Lie group G contained in its complexification 
Gq. It is important to have a maximal domain, and towards this goal we show 
that this one is maximal in some directions. 
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author was supported in part by NSF grant DMS-0070742. 
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Although defined in terms of subgroups of Gc , the domain is natural also 
from the geometric viewpoint. This theme is developed more fully in [KrStll] 
where we show that the quotient of the domain by Kq is bi-holomorphic to a 
maximal Grauert tube of G/K with the adapted complex structure, and where 
we show that it also contains a domain bi-holomorphic but not isometric with a 
related bounded symmetric domain. Some implications of this for the harmonic 
analysis of G/K are also developed there. 

However, the main goal of this paper is to use the holomorphic extension 
of iT-finite vectors and their matrix coefficients to obtain estimates involving 
automorphic functions. To our knowledge, Sarnak was the first to use this 
idea in the paper [Sa94]. For example, with it he obtained estimates on the 
Fourier coefficients of polynomials of Maafi forms for G = SO(3, 1). Sarnak also 
conjectured the size of the exponential decay rate for similar coefficients for 
S1(2,R). Motivated by Sarnak's work, Bernstein-Reznikov, in [BeRe99], veri- 
fied this conjecture, and in the process introduced a new technique involving 
G- invariant Sobolev norms. As an application of the holomorphic extension of 
representations and with a more representation-theoretic treatment of invari- 
ant Sobolev norms, we shall verify a uniform version of the conjecture for all 
real rank-one groups. As the representation-theoretic techniques are general, 
we are able also to obtain estimates for the decay rate of Fourier coefficients 
of Rankin-Selberg products of Maafi forms for G = Sl(n, R), and to give a 
conceptually simple proof of results of Good, [Go81a,b], on the growth rate of 
Fourier coefficients of Rankin-Selberg products for co-finite volume lattices in 
S1(2,R). 

It is a pleasure to acknowledge Nolan Wallach's influence on our work by 
his idea of viewing automorphic functions as generalized matrix coefficients, 
and to thank Steve Rallis for bringing the Bernstein-Reznikov work to our 
attention, as well as for encouraging us to pursue this project. To the referee 
goes our gratitude for a careful reading of our manuscript that resulted in the 
correction of some oversights, as well as a notable improvement of our estimates 
on automorphic functions for S1(3,R). 

1. The double coset domain 

To begin we recall some standard structure theory in order to be able 
to define the domain that will be important for the rest of the paper. Any 
standard reference for structure theory, such as [Hel78], is adequate. 

Let g be a real, semisimple Lie algebra with a Cartan involution 9. Denote 
by = £ © p the associated Cartan decomposition. Take a C p a maximal 
abelian subspace and let S = S(g, a) C a* be the corresponding root system. 
Related to this root system is the root space decomposition according to the 
simultaneous eigenvalues of &d(H),H E a : 
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= aeme 0g a ; 

here m = j e (o) and g° = {X G g: (Vtf G a) [iZ",X] = a(H)X}. For the 
choice of a positive system S + C S one obtains the nilpotent Lie algebra 
n = ae s+ a - Then one has the Iwasawa decomposition on the Lie algebra 
level 

= teo0n. 

Let Gc be a simply connected Lie group with Lie algebra gc, where 
for a real Lie algebra I, by ic we mean its complexification. We denote by 
G, A, Ac, K, Kc, N and Nc the analytic subgroups of Gc corresponding to 
g, a, ac, 6, n and tic- If u = 6 © then it is a subalgebra of gc and the 
corresponding analytic subgroup U = exp(u) is a maximal compact, and in 
this case, simply connected, subgroup of Gc- 

For these choices one has for G the Iwasawa decomposition, that is, the 
multiplication map 

K x A x N — > G, (k, a, n) i-> fcan 

is an analytic diffeomorphism. In particular, every element g G G can be 
written uniquely as g = n{g)a{g)n(g) with each of the maps n(g) £ K, 
a(g) G A, n(g) G ./V depending analytically on g G G. 

We shall be concerned with finding a suitable domain in Gc on which this 
decomposition extends holomorphically. Of course, various domains having 
this property have been obtained by several individuals. What distinguishes 
the one here is its K<c-G double coset feature as well as a type of maximality. 
First we note the following: 

Lemma 1.1. The multiplication mapping 

<&: Kc x ^4 C x Nc — > Gc, (k, a, n) i-> kan 
has everywhere surjective differential. 

Proof. Obviously one has gc = 6c © ac © n;- and ac © nc is a subal- 
gebra of gc- Then following Harish-Chandra, since $ is left Kc and right 
iVc-equivariant it suffices to check that d<3?(l,a, 1) is surjective for all a G Ac- 
Let p a (g) = ga be the right translation in Gc by the element a. Then for 
X G ic, y £ <*C and Z G tic one has 

d$(l, a, 1)(X, Y, Z) = dp a (l)(X + Y + Ad(a)Z), 
from which the surjectivity follows. □ 

To describe the domain we extend a to a ^-stable Cartan subalgebra t) of 
g so that f) = a © t with t C m. Let A = A(gc, f)c) be the corresponding root 
system of g. Then it is known that A| a \{0} = S. 
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Let II = {cci, . . . , a n } be the set of simple restricted roots corresponding 
to the positive roots S + . We define elements u\, . . . , u> n of a* as follows, using 
the restriction of the Cartan-Killing form to o: 

(ujj,ai} = iii^j 



(VI <i,j< n) 



feS = l if«^Aand2a^S 
- 2 if on & A and 2oh G E. 



Using standard results in structure theory relating A and S one can show 
that ui,...,u; n are algebraically integral for A = A(gc,f)c)- The last piece 
of structure theory we shall recall is the little Weyl group. We denote by 
W a = N K (a)/Z K (a) the Weyl group of E(o,fl). 

We are ready to define a first approximation to the double coset domain. 
We set 



and 



a 1 c = {X £ a c : (VI < k < n)(\/w G W a ) I Tmu k (w.X)\ < ^} 

a<C = 2a c- 



On the group side we let 4° = exp(o^) and A c = exp(a^). Clearly W a leaves 
each of a^, a' . 4^ and 4^ invariant. 

If a € a c is analytically integral for 4c, then we set a a = e a ( loga ) for 
all a £ 4c Since Gc is simply connected, the elements ujj are analytically 
integral for 4c and so we have a^ k well defined. 

Next we introduce the domains 

4°'- = {a G 4 C : (VI < jfe < n) Re(a Wfc ) > 0}, 

and 

4^ = (4^)5 = {a G 4c: (VI < k < n)\ arg(o w *)| < ^}. 
Note that 4^ C 4°'- and Aj, C 4^'-. 



Lemma 1.2. (i) For O C 4c open, KcQNc open zn Gc- /n particular, 
the sets K C A C N C , K c A l c N c , K C A^-N C , K C A° C N C and K C A^-N C are open 



in Gc. 

(ii) KcAcNc is dense in Gc- 

Proof. This is an immediate consequence of Lemma 1.1 as $ is a morphism 
of affine algebraic varieties with everywhere submersive differential. □ 

Proposition 1.3. Let Gc be a simply connected, semisimple, complex 
Lie group. Then the multiplication mapping 

<£: K x .1°'' x N c — > Gc, (k, a, n) i-> kan 

is an analytic diffeomorphism onto its open image KcA c '~Nc- 
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Proof. In view of the preceding lemmas, it suffices to show that <E> is 
injective. Suppose that kan = k'a'n' for some k,k' G Kc, a, a' G A c '~ and 
n,n' G Nq. Denote by the holomorphic extension of the Cartan involution 
of G to Gc- Then we get that 

Qikan^kan = S(k' ' a' n')' 1 k'a'n' 

or equivalently 

e(n- 1 )a 2 n = Q((n')- 1 )(a') 2 n'. 

Now the subgroup Nc = @(Nc) corresponds to the analytic subgroup with 
Lie algebra nc = © ae _£+ flc- As a consequence of the injectivity of the map 

N c x A c x N c -> N c AcN c , (n, a, n) i-> nan 

we conclude that n = n' and a 2 = (a 1 ) 2 . We may assume that a, a' G exp(io). 

To complete the proof of the proposition it remains to show that a 2 = (a') 2 
for a, a' G -A c '~ implies that a = a'. Let X\, . . . , X n in ac be the dual basis to 
ui, . . . ,u n . We can write a = exp(^" =1 y>jXj) and a' = exp(J^" =1 f'jXj) for 
complex numbers ifj, ip'j satisfying | Im ipj\ < §, | Im^| < ^. Then a = (a') 2 
implies that 

e 2 K = a 2u> 3 = (a / )2 ^. = e 2^ 

and hence ifj = ip'j for all 1 < j < n, concluding the proof of the proposition. 

□ 

Thus every element z G KqAq-Nq can be uniquely written as z = 
K(z)a(z)n(z) with k(z) G Kc, a(z) G A^~ and n(z) G Nc all depending 
holomorphically on z. Next we define domains using the restricted roots. We 
set 

b° = {X G a: (Va G E) |a(X)| < vr}. 

and 

Clearly both b° and b 1 are Wa-invariant. We set b c = a+ib J and i? c = exp(b c ) 
for j = 0, 1. Let o° = i(a c Did). Then, from the classification of restricted root 
systems and standard facts about the associated fundamental weights, one can 
verify that a C b°. For a comparison of these domains we provide below the 
illustrations for two rank 2 algebras. 

Lemma 1.4. Let to C ib 1 be a nonempty, open, Wa-invariant, convex set. 
Then the set 

Kc exp(u)G 



is open in Gc- 
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Figure 1 Figure 2 

Figure 1 corresponds to sl(3,M) and Figure 2 to sp(2,M). The 
region enclosed by an outer polygon corresponds to b° while that 
enclosed by an inner polygon corresponds to a . The H ai denote 
the coroots of on and we identify the iVi as elements of a via the 
Cartan-Killing form. 

Proof. Set W = Ad(K)uj. Since uj is open, convex, and W a -invariant, 
Kostant's nonlinear convexity theorem shows that W is an open, convex set 
in ip. Note that K c exp(u)G = K c exp(W)G. Now [AkGi90, p. 4-5] shows 
that the multiplication mapping 

to: Kc x exp(VF) x G — > (! ■. (k,a,g) t— » kag 

has everywhere surjective differential. From that the assertion follows. □ 

For each 1 < k < n we write (iT k ,V k ) for the real, finite-dimensional, 
highest weight representation of G with highest weight u k . We choose a scalar 
product (•, •) on V k which satisfies (ir k (g)v,w) = (v,ir k (@(g)^ 1 )w) for all v,w G 
V k and g G Gc- We denote by v k a normalized highest weight vector of (ir k , V k ). 

Lemma 1.5. For all 1 < k < n, a £ A^, and m G N, 



Re((7r fc (#(m) 



-1 2 \ 

a m)v k ,v ki 



> 0. 



Proof. Fix 1 < A; < n, a and to G N, and note that a 2 G A^. Now, 

(1.1) (n k {0{m)~ 1 a 2 m)v k ,v k ) = (ir k (a 2 )n k {m)v k ,ir k (rn)v k ) . 

Let Pfc C o* denote the set of o-weights of (ir k ,V k ). Then (1.1) implies that 
there exist nonnegative numbers eg, (3 G V k , such that 

(Tt k (6(m)- 1 a 2 m)v k ,v k ) = ^ c f3 a 2/3 . 

3eP k 
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Recall that 

V k C conv(Wa^fc). 

Since a c is convex and Weyl group invariant, to finish the proof it suffices 
to show that Re(a 2uJk ) > for all a G A c . But this is immediate from the 
definition of o^. □ 

Lemma 1.6. Let (bj)j^ be a convergent sequence in Ac and (nj)jeN om 
unbounded sequence in Nc- Then the sequence 

( e K)~V.). 6N 

is unbounded in Gc- 

Proof. Let d(-, •) be a left invariant metric on Gq. Then 
d(e(nj) _1 ^nj, 1) = d(b)n h @(nj)), 

and we see that lim^oo d(@(nj)~ 1 b 2 nj, 1) = oo (this follows for example by 
embedding Ad(Gc) into Sl(m,C), where we can arrange matters so that Aq 
maps into the diagonal matrices and Nc in the upper triangular matrices) . □ 

Proposition 1.7. (i) K C A C G is open in Gc- 

(ii) K C A C G C K C A C ^N C . 

(iii) For all X £ a c the mappings 

A c xG^C, (a,g) ^ a(ag) x , 

A c xG^K c , (a,g)^K(ag) 

are analytic, and holomorphic in the first variable. 

Proof, (i) appears in Lemma 1.2. For (ii) take an a G A c . First we show 
that gl/V C K c A c N c . Fix m G iV and let 

O = {a G A^: am G i^cA^c} 

= {a G .1' : &(my 1 a 2 m G iVc^c^c}- 

Then Q is open and nonempty. We have to show that O = A c . Suppose the 
contrary. Then there exists a sequence (aj)j^ in Q such that ao = lim^oo aj G 

Let a G f2. Then by Proposition 1.3 we find unique fc G Kq, b G Ac and 
n G A^c such that am = kbn or, in other words, 

e(m) _1 a 2 m = 0(n) _1 & 2 n. 

Taking matrix-coefficients with fundamental representations we thus get that 

(1.2) b 2 ^ = (^(BH- 1 ^, u fc ) = (TTfcCeM-^^)^, Ufc) 
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for all 1 < k < n. Applied to our sequence (aj)j^ we get elements kj £ Kc, 
bj £ Ac and rij £ Nc with ajm = kjbjUj. Lemma 1.5 together with (1.2) 
imply that (bj)j£^ is bounded. If necessary, by taking a subsequence, we may 
assume that bo = hin^oo bj exists in Ac- Since Q(m)~ 1 aQm N^A^Nc, 
the sequence (uj)j^ is unbounded in Nc- Hence (@(nj)~ 1 bjrij^J ^ is an 

unbounded sequence in Gc by Lemma 1.6. But this contradicts the fact that 
(^d(m)~ 1 a'jmj ^ is bounded. Thus we have proved that aN C KcAcNc 

for all a £ A c . But now (1.2) together with Lemma 1.5 actually shows that 
b £ .l 1 .' . hence aN C KcA c '~Nc for all a £ A c . The Bruhat decomposition 

of G gives G = {J weWa NwMAN with M = Z K (A). Since A c is N K (A)- 
invariant, we get that aG C KcA^-Nc- Then (ii) is now clear while (iii) is a 
consequence of (ii) and Proposition 1.3. □ 

Next we are going to prove a significant extension of Proposition 1.7. We 
will conclude the proof in the following section. 

Theorem 1.8. Let G be a classical semisimple Lie group. Then the 
following assertions hold: 

(i) KcB c G is open in Gc', 

(ii) r>U; c i^cAfc 

(iii) there exists an analytic function 

B c xG^a c , (a,g)^H(ag), 

holomorphic in the first variable, such that ag £ Kc exp H(ag)Nc for all 
a £ B c and g £ G; 

(iv) there exists an analytic function 

k: Be x G -> K € , (a, g) h-> n{ag), 

holomorphic in the first variable, such that ag £ n(ag)AcNc for all a £ 
B c and g £ G. 

Proof, (i) follows from Lemma 1.2. (ii) follows from Proposition 2.5, 
Proposition 2.6 and Proposition 2.9 in the next section. 

(iii) Set L = Kc n Ac and note that L is a discrete subgroup of Gc- Then 
the first part of the proof of Lemma 1.3 shows that we have a biholomorphic 
diffeomorphism 

(K c x L Ac) x iV c -> KcAcNc, ([k, a],n) h-> kan. 
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In particular, we get a holomorphic middle projection 

a: KqAi^Nq — > Ac/L, kan ^ aL, 

and so, by (ii), an analytic mapping 

3>:B c x G ^ A c /L, (a,g)^a(ag). 

Now o;- — > Ac/L, via the map X exp(X)L, is the universal cover of Ac/L. 
To complete the proof of (iii) it remains to show that <E> lifts to a continuous map 
with values in ac- Since exp: — ► A c is injective, Proposition 1.7 implies that 
^ I A 1 xG to a continuous map \1/ with values in ac- Since the exponential 
function restricted to b c is injective (cf. Remark 1.9.), B c is simply connected 
and so for every simply connected set U C G we get a continuous lift of 3> [b^ X [/ 

extending \I/ U^xC- By the uniqueness of liftings we get a continuous lift of <I> 
completing the proof of (iii). 

(iv) In view of (ii), we get an analytic map 

k:B c xG -> K c /L, (a,g)^K(ag) 

even holomorphic in the first variable and such that ag € n(ag)AcNc- Thus 
in order to prove the assertion in (iv), it suffices that k lifts to a continuous 
map k: B c xG-» Kc- But this is proved as in (iii). □ 

Remark 1.9. The simply connected hypothesis on Gc that has been made 
is not necessary. More generally, if G is classical, semisimple and contained in 
its complexification, then Theorem 1.8 is valid. Indeed, let g be a semisimple 
Lie algebra with Cartan decomposition q = I © o © n, 0c its complexification 
and let Gc be a simply connected Lie group with Lie algebra 0c- As before, 
let G be the analytic subgroup of Gc with Lie algebra 0. 

Let now G\ be another connected Lie group with Lie algebra and suppose 
that G\ sits in its complexification G^c- Write G\ = K\A\Ni for the Iwasawa 
decomposition of G\ corresponding to q = fiffio©n. SetB\ c = A\ exp Gi c (^ 1 )- 
Since Gc is simply connected, we have a covering homomorphism 

7r: Gc — > G\ : c- 

Hence Theorem 1.8 (ii) implies that 

B\ C G\ C Ki^Ai^Ni^c- 

To see that Theorem 1.8 (iii), (iv) remains true for G± contained in Gi c one 
needs that B\ c is simply connected. But this will follow from the fact that 
ex PG lC : ^ ^ic i s injective. To see this, note that this map is injective if 
and only if the map 

f-.b 1 Ai iC , X»exp Gifi (X) 
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is injective. If / were not injective, then there would exist an element X G b , 
X ^ 0, such that exp GiC (A") = 1. Hence a(X) G i2nZ for all a G S 

(cf. [Hel78, Ch. VII,§4, Prop. 4.1]), a contradiction to X G b°\{0}. 

The next proposition will be used in a later section. It has independent 
interest as it can be considered as a principle of convex inclusions and as such 
is related to Kostant's nonlinear convexity theorem. 

Suppose that E is a subset in a complex vector space V . We denote by 
conv E the convex hull of E and by cone E = M + E the cone generated by E. 

PROPOSITION 1.10. Let G u C b° be a connected subset. Set b£ = a+iu 
and B£ = exp(bg) . Then, 

1>>~C C i^c^ciVc => B£ onva; G C KcA^c- 

Proof. Fix g £ G. It suffices to show the existence of a holomorphic 
function 

/^r^ac, a^f g (a) 

such that a# G K c exp(f g (a))N c for a G B™ nvu} holds. We already know 
from Theorem 1.8 (hi) that a holomorphic function f g '-B£ — ► etc with ag G 
K c exp(^(a))iVc for a £ B£ exists. Now 5^ onva; is the holomorphic hull of 
B£ and so / 9 extends to a holomorphic mapping / 9 : B™ nvuJ — > etc- 

It remains to show that ag G Kcex.p(f g (a))Nc for a G i?g onva; . If not, 
then we find a convergent sequence (a n ) nG N with liim^oo a n = ao G i?£ onvw , 
a n g G -fTcAc-^C but 005 ^ -fTcAc-^c- Hence we find a sequence m n G iV"c such 
that 

®(g)~ l a 2 n g = O^n)" 1 f g (a n ) 2 m n 

but @(g)~ 1 a 2 ) g G" iVc^c^c- As (^f g (a n )j ^ is bounded, we conclude (cf. 
Lemma 1.6) that (m n ) nG N is unbounded, a contradiction. □ 



2. Matrix calculations 

We shall prove (ii) of Theorem 1.8 by various results about matrices. First 
we shall treat the group G = Sl(m,lR), m > 2. Then we shall give a class of 
subgroups of Sl(m, R) whose roots have a hereditary property similar to one 
held by Levi factors of parabolic subgroups. This will allow us to take care 
of most of the classical groups. The remaining cases are treated at the end of 
this section. 

Here we obviously have G C Gc = Sl(m, C) with Gc simply connected. 
We let i = so(m,M.) and choose 
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= {diag(xi,x 2 , . . . ,x m ) e M(m,R):^2xi = 0} 

i=i 

as a maximal abelian subalgebra in p = Symm(m, K) r\sl(m, M). Define ele- 
ments £j £ a* by setting 

£j(diag(xi, . . .,x m )) = Xj. 

Then £ = {si — Eji 1 < i ^ j < m} and we take S + = {gj — ey.i < j} as a 
positive system. The associated system of simple restricted roots is given by 

II = {ex — e 2 , . . . , e m _i — e m }. 

As q is split we have £ = A. In particular, the ujj, 1 < j < m — 1 are the usual 
fundamental weights and are given by 

ujj = e± + . . . + £j, (1 < j < m — 1). 

The Weyl group W a of S(o, g) is the group of permutations on the m elements 

£l j • • • j £m- 

In matrix notation the nilpotent groups N and N are given by: 



N = { 



and 



iV = { 



( 1 X12 

1 ^23 

V 

/ 1 

a?2i 1 



X\m ^ 
^2m 



\ X m \ . . . X m ,m—1 1 / 

For each 1 < j < m we set ej = ($k-j,i-j)k,i G diag(m,R). Further we 
date to each u>j the element X Wj = J2i=i e J ~ m ^j=i e r 

Lemma 2.1. (i) 



b° = int ^ conv ({±7ru>.X Wj : u; G W a , 1 < j < m - 1})^ ; 



(ii) 



m — 1 



j'=i 



Proof, (i) Set 

b' = int ^ conv ({±7raX Wj : w G W a , 1 < j < m - 1})^ . 
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Both b° and b' are closed, convex and Wa-invariant. Thus by the convexity 
of b° and b' we have to show only that b° = b'. Now W a rotates the extreme 
points of both b° and b', and the extreme points of b' are given by iw.wX^.. 
We shall prove the result by double containment. 

"D" : By the Krein-Milman Theorem it suffices to show that ±irX Wj G b° 
for all 1 < j < m— 1. Every a G S + can be written as a = Y^Y=i ^j{ £ j ~ £ j+i) 
with coefficients Sj G {0,1}. Thus a(X Wj ) G {0,1} and the inclusion "D" 
follows from the definition of b°. 

"C": Notice that ui, . . . ,co m -i constitute a basis of a*. Hence every 
X G b° can be written as X = X^j^ 1 with coefficients Aj G R. From the 

definition of b' we may assume that Aj > for all 1 < j < m— 1. In particular, 
we see that 

m—1 m—1 

(ei - e m )(X) = ]T fe" " = E IM> 

j=i j"=i 

concluding the proof of "C" . 

(ii) For X = diag(xi, . . . , x m ) = YJjLi xjej G b°, 

-7T < 2X1 + ^2 + . . . + X m = Xl - X m < IT, 

and 

— 7T < xi — Xj < 7T for all 2 < j < m — 1. 

By summing these inequalities we obtain 

— (m — 1)tt < mx\ < (m — l)ir, 

or equivalently |xi| < ^^-vr. Similarly, \xj\ < 22 ^-vr for all 1 < j < m. □ 

Remark 2.2. Notice that a is strictly smaller than b°, although they 
have common boundary points (cf. Figure 1). In particular, Lemma 2.1 shows 
that 

da n db° D {|(e; - ej ): 1 < i + j < m - 1}. 

For every 1 < k < m we denote by Ak(A) the k th principal minor of a 
matrix A G M(m, C). For every g = (gij)i<i t j< m £ M(m, C) and 1 < k < m 
we define 5 (fc) G M(fe,C) by = {gij)i<i,j<k- 

Proposition 2.3. lei G = Sl(m,R) twito G c = Sl(m,C). T/ien for all 
1 < fc < m, a G and 5 G Sl(m,M) mi/i g^ G G1(£;,R), 



(i) A fc ( 5 a<7*) + 0; 

(ii) Spec {{gag 1 )^^ C cone ^ conv(Spec(a))) . 
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Proof, (i) Fixing 1 < k < m, a G B° and g G Sl(m, C) with 
flf( fc ) G Gl(fc,C), we write a = diag^ie^ 1 , . . . ,r m e^ m ) with > 0, -^vr < 
V< < (cf. Lemma 2.1(h)). Set 

with 5(fc) G Gl(ife,R) and 5 G M(k x(m-k),R). Then, 

A fe (W) = ^((^ ^diag(r ie ^,...r m e^)^W 
= det fc (g {k) diag(rie^ , . . . , r fc e^* )<?f fc) 

+5 diag(r fe+1 e^ fe + 1 , . . . , r m e^)B*) . 

In order to show that A k {gag l ) / Owe have to show that the k x fc-matrix 
X [k) = g [k) diag(ne^, . . . ,r k e^)g\ k) + B diag(r fe+1 e^+\ . . . ,r m e**»)B' 
is invertible. 

Assume first that k < m — k. Then we can write B = (Bi,B 2 ) with 
B x G M(k,R) and B 2 G M(fe x (m - 2&),R). Hence we obtain that 

B diag(r fe+1 e^ fc+1 , . . . , r m e^™)B< = B x diag(r fe+1 e^ fc + 1 , . . . , r 2fe e^ 2 * )B* 

+B 2 diag(r 2fe+1 e^ fc+ i , . . . , r m e^)B*. 

Let (■, ■) be the usual hermitian inner product on C k . In particular, if 
v G C fe , f / 0, then we get 

<X (fc) ^> = {&s,g{r 1 e i *\...,r k e i ^)g\ k) v,g\ k) v) 

+<diag(r fe+1 e^ fc+1 , . . . , r 2fe e^)B^, B* V ) 
+(diag(r 2fe V 2fc+1 , . . . , r m e^)B^, 
So there exist numbers ci, . . . , c rn > 0, not all zero, such that 

m 

(2.1) (X {k) v,v)=Y,c j e i ^. 

i=i 

Similarly one shows that (2.1) holds for the case k > m — k. Now (i) follows 
from (2.1) and Lemma 2.4 below. 

(ii) Since = (gag 1 )^, (ii) follows from (2.1). □ 

We denote by C + = {z G C: z G"] - oo, 0]} the split plane in C. 
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Lemma 2.4. Let pi, . . . ,<p m £ K 6e suc/i i/iai diag(<£>i, . . . , p m ) G b°. 
Then for all sequences of nonnegative numbers c±, . . . , c m , not all zero, 

m 
3=1 

In particular 2~Zj=i Cje 1 ^' / 0. 

Proof. As b° is Wa-invariant there is no loss of generality to assume that 
(pi < . . . < p m . Then < pj — pi < it for all 1 < j < m. Since Sj=i Pj = 
we have p m > 0. Thus^jLi Cj^ 3 is a sum of vectors not all zero in the real 
convex cone 

C = {z G C: ip m - tt < arg(z) < ip m } 

in C. In particular Y^J L =i c j elipj ls nonzero since the convex cone C is pointed 
(i.e. contains no affine lines). Since < p m < — ^-vr (cf. Lemma 2.1(h)) we 
also have C\{0} C C + , concluding the proof of the lemma. □ 

Proposition 2.5. For G = Sl(m, R), 

K C B C G C K C A C N C . 

Proof. Take a G B^ and recall that a 2 £ 5j. First we show that aN C 
KqAqNc. Let n G N. Then Proposition 2.3(i) says that all principal minors of 
the complex symmetric matrix n*a 2 n are nonzero. Hence a theorem of Jacobi 
(cf. [Koe83, p. 124]) implies that there exist unique elements bo G Ac and 
m G iVc such that 

n*a 2 n = mfborn. 

Let ao G ^4c be such that a 2 , = &o- Then we have 

an = kaom 

with k G ifc given by k = anm~ l Oq 1 . 

Using, as before, the Bruhat decomposition G = LLeW a NwMAN, to- 
gether with the iVx(^4)-invariance of B c , we get that aG C ifcAc-^c for all 
g £ G, completing the proof. □ 

With G = Sl(m, R) out of the way we want to use an observation that 
will allow us to obtain a proof of Theorem 1.8(h) for appropriate subgroups. 
The groups that will be covered in this way are: Sp(n, R), Sp(p,q), Sp(n,C), 
SV(p,q), SO*(2n), Sl(n,C) and Sl(n,H). 

Recall that a Levi subalgebra m of a standard parabolic subalgebra must 
be of the form m = m(0) for O C II. If, moreover, m is ^-stable, then the 
Iwasawa decomposition for m is compatible with that of 0. More generally, 
for g = sl(m,R) we consider ^-stable subalgebras gi C g with a property 
that will give them Iwasawa decompositions compatible with that of q. Set 
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f?i = t PI 0i and pi = p PI 0i so that 0i = 6i © pi is a Cartan decomposition 
of 0. Let oi C pi be a maximal abelian subspace. Since we can extend oi to 
a maximal abelian subspace of p and since all maximal abelian subspaces of p 
are conjugate under Ad (if), we may assume that di C a. Choose a positive 
system T>f of Si = E(0i,ai). Then we can find a positive system E + of E 
such that E+ C E + | ai . Write ni = QgS + 0i and note that ni C n. 
We now impose the following condition on the restricted roots: 

(I) E| ai \{0} = E 1 . 

It can be checked that (I) holds for example for the standard imbeddings of 
the subalgebras 0i = sp(n,M) (with 2n = m), su(p,q) (with 2p + 2q = m), 
S P(P) 0) (with 2p + 2q = m) or so*(2n) (with 2n = m)(in all cases the fact that 
makes things work is that the restricted root system of 0i is either of type C n 
or BC n ). Further examples are 0i = sl(n, C) (with 2n = m), sp(n,C) (with 
2n = m) or sl(n, H) (with 4n = m) (here the explanation is that the root 
system Ei is of type A). Set 

bi = {X G oi: (Va G Si) \a(X)\ < vr} 

and 

b\ = -b° v 
1 2 1 

Then condition (I) guarantees that 

(2.2) b\ C b 1 . 

We denote by Gi the analytic subgroup of G which is associated to 0i. 
We assume that G\ is closed. Further we denote by K±, A±, Ni and Ni the 
analytic subgroups of G\ corresponding to t±, Oi, tti and Hi. Finally we set 
Bj c = exp(oi + ib\). In order to prove Theorem 1.8(H) for the group G\ we 
have to show that 

B^d c ^i >c Ai )C iVi ) c 

or equivalently 

(2.3) (Vb€Bi )C )(V5€Gi)(3a€ Ai, c ,m€ A^i,c), g t bg = m t am. 

In view of (2.2) and the validity of (2.3) for G we deduce that for all b € 
B{ c , g G Gi there exist unique elements m = m(b,g) G A^Ci a = «,(b,g) G 
^4c such that 6 2 = m t am. Moreover a = a(b,g) and m = m(b,g) are 
analytic functions in the variables b G B\ c , g G Gi. Since we already know 
that a(A\ c ,G\) C ^i^ and m(^4j c ,Gi) C A^i 5 c (cf. Proposition 1.7), the 
analyticity of both a and m implies that a(B\ c , Gi) C and m(Sj c , Gi) C 
Ni y€ proving (2.2). 
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We summarize the above discussion with 

Proposition 2.6. Assume that G is one of the groups Sl(n,R), Sp(n,M), 
Sp(p,g), SU(p,g), SO*(2n), Sl(n,C), Sl(n,M) orSp(n,C). Then 

K . liC C KcAcTVc- 

There remain the restricted root systems for 9 = so(p, q) and g = so(n, C). 
So first we recall some facts concerning these root systems of type B n and D n . 

B n : The root system B n is given by 

£ = {±£j ± £,: 1 < z / j < n} U {±£i: l<i<n}. 
A basis of S is 

II = {£1 — £2, £2 — £3i • • • > £n-l — £ ra; e n}- 

If is a split real Lie algebra with restricted root system S, then the u>i are 
the fundamental weights associated to II and given by 

Wl = £l, L02 = £l + £2, • • • , W n _l = £1 + . . . + £„_l, CJ„ = -(£1 + . . . + £„). 

D n : The root system D n is given by 

E = {±£i±£ i: l < i < n} 
and a basis of S is given by 

II = {£l — £2, £2 — £3 5 • • • j £n-l — £n, £n-l + £n}- 

If q is a split real Lie algebra with restricted root system S, then the tOi are 
the fundamental weights: 

LOl = £1, CJ 2 = £l + £2, • • • , W n -2 = £l + . . . + £n-2 

and 

W„-l = -(£! + ... + £ n -l - £n), W„ = -(£l + • • • + £„)• 

To indicate the dependence of b° on the root system, we shall write b°(S) 
for b°. 

Lemma 2.7. T/iere exisfe b°(D n ) = b°(B n ). 

Proof. This is immediate from the equality conv(i? ra ) = conv(D n ). □ 

The final goal of this section is to prove the inclusion 
(2.4) BqG C K c A c N c 

for G = SO(p, g) or G = SO(n, C). 
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We shall repeat the strategy used for the target group of type A n . So 
assume for the moment that (2.4) holds for G = SO(n, n). Assume that p > q 
and embed G\ = SO(p, q) into SO(p, p) in the natural way (upper left corner 
block). Then if we restrict £ to ai we get a root system of type B q or D q . Hence 
by our restriction procedure from the preceding section and Lemma 2.7, we get 

(2.4) also for the subgroup G\. Thus it suffices to prove (2.4) for G = SO(n, n) 
and G = SO(n, C), with both so(n,n) and so(n, C) split. 

In what follows q denotes either $o(n,n) or so(n, C). We embed q into 
s((2n, R) as in the previous section. Then if we restrict the weights of sl(2n, R) 
to g, we obtain a root system of type C n or BC n . We set 

b° rcs = b°(C n ) = b°(BC n ) 

and 

b 1 =-b° 

^rcs 2 res 

On the group side we define B^. cs c = exp(a + ib( cs ) for j = 0, 1. In particular 
we get that 

(2.5) B^ C G C i^c^c^c- 

We write (7r n , V n ) for the n th fundamental representation of G with highest 
weight w n = |(ei + . . . + £«)• We write P n for the set of o-weights of (n n , V n ) 
and set 

b(7r n )° = {X e a: (VaeV n ) \a(X)\ < |}. 

As usual we put b(7r n ) 1 = ^b(7r ra )°. 

Lemma 2.8. T/ie following holds: 

conv(b r ° cs U b(7r n )°) D b°. 



Proof. We claim that the extreme points of b° are given by 
Ext(P) = 



{±7rej, f (±ei ± . . . ± e n )} for n > 3, 
{±7rej} for n = 2. 



In fact we have b° = b°(B n ) by Lemma 2.7 and so Ext(b°) is invariant under 
the Weyl group W(B n ) = (2^)™ x 5" n . From that the claim follows. 

Now we have f (±ei ± . . . ± e n ) S b^ cs and ±7rej G b(7r n )°. Hence the 
assertion of the lemma follows from the Krein-Milman theorem. □ 

Proposition 2.9. Assume that G = SO (p,q) or G = SO(n,C). Then 
for all a G B^, 

k r>U; c k c a c n c . 
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Proof. From what we have already done it is enough to prove the inclusion 
for G = SO(n, n) or G = SO(n, C). By passing to a covering group if necessary 
we can also replace G by G. Set Bc(ir n ) = exp(a + ib(7r n ) 1 ). In view of 
Proposition 1.9, Lemma 2.7, (2.5) and Lemma 2.8 it remains to check that 

(2.6) B c (ir n )G C K C A C N C . 

Suppose that (2.6) is false. Then we can find a g £ G and a convergent sequence 
(oj)j e N in Bc(iTn) with lim^oo aj = a £ B c {-K n ), ©(g^ajg £ N C A C N C for 
all j 6 N but 0(g) -1 a^g NcAcNc- I n particular we find elements rrij G Nc 
and bj £ Ac with 

©(i/) -1 ^ = 6(mj) -1 &jmj. 

To arrive at a contradiction we have to show that (6j)jeN is bounded (cf. 
Lemma 2.4). Let (-, •) denote an hermitian inner product on V n with {Tr n (g)v, w) 
= (v,TT n (@(g)~ 1 )w) for all g £ Gc, v,w £ V. Let Q C V^\{0} be a compact 
subset. Then the definition of b(-7r ra )° shows that 

(2.7) inf„ 6Q Re(7r n (e( 5 )- 1 a J 2 5 ) W , v) > 0. 

If v = [r^r^4=r^ri f° r a normalized weight vector w a with weight a, we 

get 



\TT n (m j ^VaW 2 



(n n (@(mj) 1 b j m j )v,v) = {ir n (@(g) 1 a 2 j g)v,v) 



1 . 1<J! ^ 



J 

for all j £ N. In particular, (2.7) implies that there are constants C±,C2 > 
such that 

kn^" 1 )^! 

Recall that the weight spaces of the spin-representation (i: n ,V n ) are one- 
dimensional. Hence it follows that (ir(n)v a , v a ) = (v a ,v a ) for all n £ Nc and 
all weight vectors v a £ V n . For the same reason we get ||7r n (mj)(w a )|| 2 > 1 for 
all rrij. In particular we obtain that 

(2.8) (Va £ V n ) \b«\>C 

for some constant C > 0. Now we have V n = —V n and so (2.8) actually implies 
that (6j)j £ N is bounded. □ 

It seems reasonable to expect that a better technique would show Theo- 
rem 1.8 to be valid also for the exceptional groups. Thus we formulate 

Conjecture A. Let G be a semisimple Lie group with G Q Gc and Gc 
simply connected. Then 

B C G C K C A C N C . □ 
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Remark 2.10. In [KrStll] we clarify the geometry of the domain, thereby 
giving more evidence for its naturality. We show that the domain Kc\KcB^,G 
is bi-holomorphic to a maximal Grauert tube of K\G having complex structure 
the adapted one. We also show the existence of a subdomain of Kc\KcB^G 
bi-holomorphic to a Hermitian symmetric space but not isometric. 

3. Holomorphic extension of irreducible representations 

We now come to our first application of the preceding construction, the 
holomorphic extension of representations. Additional applications of this will 
be given in subsequent sections for specific situations, such as principal series 
of representations, specific groups, or eigenfunctions on (locally) symmetric 
spaces. The notation from representation theory needed for this section is 
standard and may be found explained in, say, [Kn86]. 

Notation. As per Conjecture A we shall write Q for if G is classical, 
and otherwise. 

Theorem 3.1. Let G be a linear, simple Lie group and let (tt,E) be an 
irreducible Banach representation of G. Then for any K-finite vector v G Ek, 
the orbit map 

G -> E, q i * ir(g)v 
extends to a G-equivariant holomorphic map on GQKq- 

Proof. Set V = Ek, the collection of if -finite vectors of (ir,E). 
Casselman's subrepresentation theorem (cf. [Wal88, 3.8]) gives the exis- 
tence of a (q, If )-embedding of V into a principal series representation 

(3.1) V (Ind£ min (a® \®l),H*,x) 

where P m i n = MAN is a minimal parabolic subgroup. In the next section 
we recall the standard terminology for principal series; in summary we set 
ir a; x = Indp m . n (<7 <8> A <8> 1); we write (W a , (■, -) a ) for the representation Hilbert 
space of a; we realize H a ^\ as a Hilbert subspace of L 2 (K/M, W a ), and we use 
induction from the right. 

Write H for the completion of V in H a ,\- Let us first assume that E = H. 
Fix v G V and write f v : G — ► H C 7i a ,x, g ir a ,x{g)v for the corresponding 
orbit map. Then we have for all g G G that 

(3.2) (Mg))(kM) = aig^kf-Ov^g^k)) (k G K). 

Hence it follows from either Theorem 1.8 (for Q = B^) or Proposition 1.7 (for 
Q, = Aq) that analytic continuation of (3.2) gives rise to a map 

(3.3) f v :GnK c ^C°°(K/M,W a ), (g » (kM » aig-^-^Kig^k)))) . 
Note that f v \ G = f v . 
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We claim that imf v C Ti. Write H for the orthogonal complement of 7i 
in the Hilbert space L 2 (K/M, W a ). Choose w G 7i L . In order to show that 
(w,im f v ) = {0}, we may assume that w is a infinite, continuous function on 
K/M. Consider the function 

F-.GQKc^C, g~(f v (g),w)= f aGT^-'MsGT 1 *)), w(k)) a dk, 

JK 

with the equality on the right-hand side following from (3.3). Since w is a 
bounded function, it is easy to see that F is holomorphic. Since F \q = and 
F is holomorphic we have F = 0. This concludes the proof of the claim. 

Next we show that /„ is holomorphic. Since V C C°°(K/M, W a ) is dense 
in TL and because weak holomorphicity implies holomorphicity, it is enough to 
show that for all w G V the analytically continued matrix coefficients 

-.GfLKc^C, g i-> {f v (g),w) 

are holomorphic. Again (3.3) gives that 

*v,M = / aig^kf-PiviKig^k))^))* dk (g G GOK c ) 

and the holomorphicity of f v follows. Before we can deduce the general case 

from the case E = TL we need a little more refined information on the orbit 
maps. Note that im/„ C TC°° , TC°° the G- module of smooth vectors (indeed 
im/u C Ti^, Ti^ the analytic vectors). Thus f v also induces a map /„: GflKc 
— ► Recall that the (Frechet) topology on is induced from the semi- 

norms 

H°° 3 v ^ ||dir(«)u|| («€W(flc))- 

From the explicit formula (3.2) of the induced action, one then deduces that 
/„ is continuous. In particular f v is holomorphic, since it is continuous and 
since for all w in the dense subspace V C (H°°)' the function (/„, w) = tt VjW is 
holomorphic. 

Finally we have to show how the general case follows from the case where 
E = H. We use the Casselman-Wallach globalization theorem (cf. [Wal92, 
11.6.7(2)]) which implies that the embedding (3.1) extends to a G-equivariant 
topological embedding on the level of smooth vectors: 

(7t,£°°)^Ka,^a)- 

Hence the Frechet representations (it^E 00 ) and (tt^a, W°°) are equivalent. As 
/„ was shown to be holomorphic for every v G V, the proof of Theorem 3.1 is 
now complete. □ 
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The holomorphic extension of the orbit map, g *— ► ir(g)v, raises the ques- 
tion of the dependence of ||7r(g)u|| on g. This we address in a subsequent 
section. The holomorphic extension of a representation also gives rise to a 
holomorphic extension of its i^-finite matrix coefficients. In the next section, 
we obtain estimates for the holomorphically extended matrix coefficients. 

4. Principal series representations 

Integral formulas. We shall look in more detail at the growth properties 
of the holomorphic extension of matrix coefficients of principal series represen- 
tations induced off a minimal parabolic subgroup. For now, we shall focus on 
the case of spherical principal series for two reasons: we shall use these results 
to obtain estimates on automorphic functions for locally symmetric spaces; 
the extension to the general case requires considering Eisenstein integrals and, 
albeit with many technicalities, given the holomorphic properties of the de- 
compositions in Theorem 1.8, this presents no fundamentally new difficulties. 

Set p = |X)aeE+ m a a G a* with m a = dimg". For A G a£. we define a 
vector space 

V x = {/ G C°°(G): (Vmara G MAN)(Vg G G) f{gman) = a x - p f{g)}. 

The group G acts on T>\ by left translation in the arguments, i.e., we obtain 
a representation (tt\,T>\) of G given by (ir\(g)f)(x) = f(g~ 1 x) for g,x G G, 
f G T>\. Besides this realization we shall need the standard realizations of 
these representations that are called the compact (resp. noncompact) picture. 
The compact realization has for Hilbert space 

lC x = Thk LHK) CL\K), 
while the noncompact realization has 

Nx = V^ L2 ^ ^- 2RC(A) *> c L\N, a(nT™<» dn). 

The representations (tt\,)C\) and (tt\,N\) are continuous representations of G. 
Moreover, the mapping 

/k-/hv (/gz>a) 

extends to a unitary equivalence (tt\,1C\) — > (71^, A/a), provided L 2 (K) is ob- 
tained from a normalized Haar measure on K and L 2 (N) is obtained from a 
Haar measure dn which satisfies J-j^a(n)~ 2p dn = 1. For A G ia* the represen- 
tations (ir\,lC\) and (ir\,Af\) are unitary. We will write (TT\,Tt\) if we do not 
want to emphasize a particular realization. 

We recall that for (tt,H) a continuous representation of a Lie group G 
on some Hilbert space H, a vector u G H is called analytic if the orbit map 
f v :G ^H, g i-> ^(g)?; is analytic. Suppose that G is contained in its universal 
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complexification Gq, and denote by g ^ g the complex conjugation in Gc with 
respect to the real form G. Then for every analytic vector v G H there exists 
a left G-invariant open neighborhood U of 1 G Gq with U = U such that f v 
extends to a holomorphic map f v :U ^ TC, g i— > Tr(g)v. With 7r* denoting the 
contragradient representation one has 

(4.1) (n(g)v,v) = {v,ir(g)*v} 

for all g & U. 

For G a Lie-group, K < G a compact subgroup, and (7r, V) a continuous 
representation of G on some topological vector space V, the representation 
(tt, V) is called K -spherical if V^" / {0}, F K = {v G F: (VA; G K)ir(k)v = v}. 
For all AeoJ- the induced representation (vr^,!^) is -FC-spherical, dimV^ = 1 
and the function 

fo:G^C, x^a{xf- p 

is a generator of . Moreover we have 

(Vg, x G G) (7r x (g)fo)(x) = a^xf-". 

In the other realizations one has vq = /o\k = Ik £ and wq = fo \ jf G AAf 
given by wo(n) = a(n) x ~ p . 

Proposition 4.1. Let (it\,1C\) be the compact realization of a spherical 
principal series representation with parameter A G and let vq = Ik G )Cf. 
Then the orbit map 

F:G^K,\, g^ir\{g)v 
extends to a holomorphic map 

F: GflK c -► K x 
on the open domain GViKc C Gc- 

Remark. Note that a slight modification of Theorem 3.1 to representations 
of finite length implies the proposition. But we shall give here a more direct 
proof avoiding the heavy machinery of representation theory. 

Proof. We consider the map 

<S>:GxK^C, (g,k) ^ aig- 1 ^-? 

and note that &\(g,-) = F(g). By Proposition 1.7 and Theorem 1.8 the 
function $ extends to an analytic map 

$:GnK c x K -»C, (z,k) ^ a{z~ 1 k) x - p 

which is holomorphic in the first argument. It is obvious that <t(z, •) G L 2 (K) 
for all z G GUKq. Let P:L 2 (K) — ► 1C\ denote the orthogonal projection and 
define 

F:GnK c ^IC x , z^P($(z,-)). 
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Then F\q = F and it remains to show that F is holomorphic. For that however 
it suffices to show that 

is holomorphic for all / 6 T>\ \k C C°°(K). But this in turn follows from 

(F(z),f)= I a{z- l kf-Pj(k) dk 
Jk 

by the compactness of K, the continuity of <3? and the holomorphy of $(-, fc). □ 

If A 6 and (7T\,/C,\) is the induced representation realized in the com- 
pact picture, then the matrix coefficient of the K-fixed vector with itself is the 
familiar zonal spherical function, 

(4.2) ifx(g) = <7r A (^ -1 )uo,«o). 

The holomorphic extension to GQKc of 7r\((7 _1 )t>o gives a holomorphic 
extension of the matrix coefficient ip\(g). However, this is not the largest 
domain of analyticity for <p\(g). Since we will estimate the norm of 7T\(g~ 1 )vo 
by means of tp\ (g) , in order to obtain optimal estimates on the norm it will be 
important to have an expression that represents tp\(g) in its entire domain of 
holomorphy. In terms of the pairing in the compact realization, <f\(g) is given 
by the well-known integral formula 

<P\(g) = [ a{gk) x -P dk. 
JK 

By the ET-bi-invariance of ip\ and in light of Proposition 1.7, this defin- 
ing integral formula for the spherical function can be extended to Kc^LKc- 
But in general the integral formula need not extend to any larger domain (cf. 
Example 4.3). There are a couple of reasons for this. First, the integrand 
k i ^ a{a~ 1 k) x ~ p becomes singular if a leaves A^, and secondly, it is no longer 
possible to take holomorphic square roots (the p-exponent frequently involves 
a square root). We shall present an alternative integral formula valid on a 
domain about twice as large and this will be crucial for the estimates on the 
norm of ir\{g~ l )vQ. 

To state the result we recall the notation 0, viz. if G is classical, then 
ft = B^. and otherwise O = Aq. Consistent with this and the notation 
(resp. Aq), we use = B^ if G is classical and otherwise set Q 2 = Aq. 

Theorem 4.2. Let A 6 and (f\ be the spherical function with param- 
eter A associated to G/K. 

(i) The spherical function ip\ extends to a Kc-bi-invariant function on 
K il 2 1\ C G ■ which is holomorphic when restricted to Q 2 . 
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(ii) (V6 G A)(\/a G exp(ia) n fi), 

<f X (ba 2 )= I a(bak) x -P ■ a{ak) x -P ■ a(ak)- 2 ** x dfc. 
In particular, for A G ia* we ge£ /or a// a G exp(ia) (~l $7 
p A (a 2 ) = / |a(a&) 2(A_p) | dfc. 

(iii) (V6 G A)(Va G exp(ia) n fi), 

Lp x (ba 2 ) = _a(ban) x ~ p ■ a(an) x -P ■ a(an)~ 2RcA dn. 

JA 

In particular, for X £ ia* , for all a G exp(za) PI 0, 
VA(a 2 ) = /ja(an) 2{A - p) | dn. 

Proof, (i) It suffices to show that ip\ \a extends to a holomorphic function 
on Q 2 . We will work with the compact realization (tv\,K.\). Let a G A. Then 
(2.1) implies that 

(4.3) (Va G A) <^ A (a 2 ) = McT 1 )^, ^(a" 1 )*^). 

We now analytically continue the right-hand side of (4.3). Recall from [Kn86, 
p. 170] that for / G T>\ and x,g G G one has 

(4-4) (ir x (gyf)(x) = a(gxr 2RcX f(gx). 

Hence ir x (g)*f = ate-r^Vx^ -1 )/ for all g G G. Similarly as in Proposi- 
tion 4.1 one shows that 5 1— ► ir(g)*vo extends to a holomorphic /CA-valued map 
on GQKc- Thus Proposition 4.1 implies that the function 

A -> C, an (7rA(a -1 )wo, ^(fiT 1 )*^) 

extends to a holomorphic function on S7. Since we have a unique holomorphic 
square root on Q 2 , namely 

S^ 2 — > ^2, a = exp(X) 1 ^ y/a = exp(-X), 

the assertion of (i) now follows from (4.3). 

(ii) In view of the proof of (i) , (ii) is immediate from the analytic extensions 
of (4.3) and (4.4) to exp(ia) n Q. 

(iii) This is proved as (ii) is by use of the noncompact realization (n\,N\) 
instead of (tt\,K.\). □ 
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Example 4.3. We explicate the theorem for the group G = Sl(2, R). 
Clearly G C Gc = S1(2,C) and G ■ is simply connected. We let 6 = so (2), 

= {(o .^xel) and n = {(° |J j:n£t}. 
For z e C*, x G C and 6> G C we set 



and 



Then 



cos# smv \ 
- sin cos / 



A° C = B° C = {a z : Re(z) > 0} and = {a z : | arg(z)| < ^}. 
Since a; = uj\ = p, we may identify with C by means of the isomorphism 

R^a*, X^Xco. 

Let us consider the spherical function with parameter A G id*. Then 
Proposition A.l(i) in the appendix shows that (4.2), the defining integral for- 
mula for (f\, extends to and we have 



(4.5) (Va z GA l c ) ipx (a z ) = ^j^ 



d6 



(z 2 +sin 2 0(4-z 2 ))2 



—A 



It is easy to see that it is no longer possible to take consistently an analytic 
square root of (z, 6) ^ z 1 + sin 2 9{\ — z 2 ) if | arg(z)| becomes larger than j. 
Also as this function has zeros, the integrand of the integral expression above 
becomes singular (although in this case the singularity is integrable). On the 
other hand Theorem 4.2(h), (hi), together with Proposition A.l imply, for all 
-f < ip < § , r > 0, that 

<p\(a re i V ) 

j_ r de 

27T J 7j- ( r 2gi V + gin 2 Q^e-iV _ r 2 e i V ^-\( e -i V + gin 2 Q^iy _ g-^))5+ A 

and 

ip x (a re i V ) -- 



dx 



+A 



If one examines the integral over K, one sees that the second factor is identi- 
cally 1 when evaluated on the real group A, so that it comes into play only on 
the complex domain. 
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An upper estimate. We can give a soft upper estimate along the convex 
hull of extreme points of the domain. 

Proposition 4.4. Let (tt x ,H x ) be a unitary principal series representa- 
tion of G. Let to be b° if G is classical and a otherwise. If a, b G to, then 

su Po<t<i^A^exp(i(ta+ (1 - t)b))j < max{(p x (exp(ia)),<p x (exp(ib))}. 

Proof. Set5 [0) i] = {z £ O.Rez £ [0,1]} and X = {za+(l-z)b: z £ S[ ,i]}. 
Then 

S'fCi] — ^ X, z i ^ g(z) = za + (1 — z)b 

defines a bi-holomorphism of complex manifolds with boundary. We set 

/: S [0A] -> C, ^ i ^ 99 A (exp(i5(z))). 

Then / is holomorphic on int <S[o,i] and we claim that / is bounded. In fact, 
we have 

f(z) = {n x (exp(lm g(z)))TT X (exp(-i^Re g(z)))v ,7r x (eyip(-i^Re g(z)))v ) , 

and so by the unitarity of tt\ 

\f(z)\ < (7r A (exp(-^Re g(z)))v ,ir x (exp(-i-Re g(z)))v Q ) = f(Rez). 

This implies our claim and so the assertion of the proposition follows from the 
Phragmen-Lindelof principle. □ 

A radial lower estimate. A precise estimate of the nature of the singular- 
ity along the entire boundary of f2 appears difficult. However, for an approach 
to the boundary along the direction of roots (or co-roots), we can obtain esti- 
mates. In this regard see Remark 5.5. We recall our standing hypothesis that 
G is a semisimple Lie group contained in its complexification Gt> 

Let H £ a and assume that there is a ^-invariant sl(2, R)-triple 
{H, X, 6(X)} C 0, i.e., [H, X] = 2X, [H, 6(X)] = -20(X), [X, 0(X)] = H. 

We shall want to give estimates for the radial behaviour 

<^(ex P (^(l -£)#)) 

for e — > provided that £ db° and (tt x ,H x ) is unitarizable. We do this 
by restriction of the representation to a subgroup isomorphic to Sl(2, R) or 
PS1(2,R). The triple gives 

go = sp&n R {H,X,6(X)}, 
a ^-stable subalgebra of isomorphic to sl(2,R). Its Cartan decomposition 
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is given by 0o = © Po with to = t go and po = p n go- Further, we set 
do = M.H. We denote the analytic subgroups of G corresponding to Qq, clq and 
6 by G , A) and K . 

We write Go for the set of equivalence classes of unitary irreducible rep- 
resentations of Go (the unitary dual of Go). Since Go is semisimple and thus 
of type I, there is a natural Ti-topology on Go, the hull-kernel topology, which 
we denote by Thk, and a Borel measure \i on Go such that 

(vr A bo, H A ) = ( ® 1 M°)> £ K a ®V a dfi{a)j . 

Here (ir a ,H a ) denotes a representative of a G Go- In particular, vo G 7if 
disintegrates as 

vo = L Vq dfi(a) 
JGo 

with 1 = ||fo|| 2 = Jg H^oll 2 d[i(a). As each Vq is Ko-fixed, for all a with 

(V<7 G Go) ^G?) = raM^KX) 

IFo II 

defines a spherical function on Go- 

In particular we get for all a £ Ao 

(4.6) 

<px(a) = (7rA(a _1 )u ,uo) = L (^(a -1 )^ , «o) "M ") = L <P?(a)IK II^M ")- 

J Go J Go 

Proposition 4.5. Let G be a semisimple Lie group with Lie algebra g 
and assume that G C Gc- Suppose that {H, X,9(X)} with H G a forms an 
sl(2,M.)-triple in q. Assume that G 9b° and £/ia£ (7Ta, Ha) unitarizable. 
Then we have 

<p x (exp(i^(l-e)H)) > C|loge| 

/or < e < 1 and a constant G > 0. 

Proof. Write Y"o = {a G Go:ir a is K-spherical}. From the well-known de- 
tails of the unitary dual of Go we know that there is a natural parametrization 
iRU]0, 1[— ► Yq. Moreover if we equip iRU]0, 1[ with its Euclidean topology, 
then this parametrization becomes continuous (this essentially follows from 
the fact that the assignments a t— ► <p®(a) are continuous with respect to the 
Euclidean topology; cf. the technique of [Wal92, 14.12.3]). In particular this 
parametrization induces the (possibly) stronger Euclidean topology r e on Yo. 
Hence if UneN Qn 1S an exhaustion of (Yo,r e ) by compact sets, then U re eN Qn 
defines an exhaustion of quasicompact Borel sets of (Yo,Thk)- So we can find 
a Q n with fg n Vq d^(cr) / 0. It follows from (4.6) that we have 

¥> A (exp(4(l - e)H)) = [ ^(exp(i^(l - e)H))\K\\ 2 M°) 
2 Jyq 2 
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for e > 0. In particular we get that 

<p x (exp(i^(l-e)H))> / ^(exp(^(l-e)F))|K|| 2 ^) 
2 JQ n 2 

for e > 0. Since Q n is compact, Theorem 5.1 (to follow) implies that there is 
a constant C > such that <^°(exp(z^(l — e)H)) > C\ \oge\ for all cr G Q n . 
Hence we get that 

^ A (exp(4(l - e)H)) > C\ \oge\ f |K|| 2 dfi(a), 
2 JQ n 

proving the theorem. □ 

Set X = {a G X: 2a £}. Let a G S and il a G a be the corresponding 
co-root, i.e., G [0 a ,0~ a ] H a such that a(H a ) = 2. 

COROLLARY 4.6. Suppose that G is one of the groups Sl(n,M), Sl(n, C), 
Sl(n,H), Sp(n,R), SO*(2ra) or SXJ(p,q). Let a G So and H a be its co-root. 
Assume that (ir\,7ix) is unitarizable. Then there exists a constant C, depend- 
ing only on A, such that 

<p x {exp(i^{l - e)H a )) > C\\oge\ 

for < e < 1. 

Proof. Since all restricted root systems are either of type A n , C n or BC n 
we have |i? Q G db°. Then the assertion follows from Proposition 4.5. □ 



5. Real rank one 

Singularity of spherical functions. We consider Lie algebras of real rank 
one. For these we are able to obtain sharp asymptotic behaviour of the holo- 
morphically extended spherical functions. 

As q has real rank one, dim a = 1. As is the custom, we set p = dimg a , 
q = dim 2a , and c = Here n = Q~ a 9 ~ 2a and [ Q ~ a , g~ 2a ] = {0}. 

We have the familiar formula for all n = exp(A + Y) = exp(A) exp(Y), 
X G Q- a , Y G fl~ 2a : 

(5.1) a{n) p = [(1 + c||A|| 2 ) 2 + 4c||y|| 2 ] E ¥ 2 . 

Here ||^|| 2 = —k(Z, 9Z) for all Z G Q with k denoting the Cartan-Killing form 
of g. Computations involving a(n) p have appeared many times. We include the 
following computations only because they involve the holomorphic extension 
of a(n) p and for this we have no convenient reference. 
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Let A a G a be defined by a(A a ) = 1. For convenience we shall identify 
ac and with C by means of the isomorphisms 

C — ► oc, zi-> zA a , 
C — > a c , A i— > Aa. 

Thus, v?A(e z ) := </?a( exp zA a ). 

Let Qg C o denote b° if g is classical and o° otherwise. 

Here and henceforth we use the notation /(e) x g(e) for two positive 
valued functions /(e), g(e) if there exist constants C\,c% > such that ci/(e) < 
<?(e) < C2/(e) for all e. 

Theorem 5.1. Lei G be a connected Lie group of real rank one contained 
in its universal complexification Gq. 

(i) For all A G ia* f/ie maximal tube domain of definition of (p\ o exp^ zs 
given by 

^A,max = 

(ii) For X G <9b°, and a fixed A G a* c , there exist a C > stic/i i/tai /or e — > 0, 
e > 



|p A (exp(±i(l-e)X)| <C 
//in addition X £ ia*, then 

\<p x (exp(±i(l-e)X)\ > 



| loge | for p = 1, q = 0, 
£ - P +i f O rp>l,q = 0, 
| loge | for q = 1, 



|loge| /or p= 1, (7 = 0, 

£ -p+i ^ or p > 1 5 g = 0, 

| loge | /or (7 = 1, 

e -5+i forq>\. 



Remark 5.2. For some A G aj^\ia* ^ can happen that Tx imax = ac; i-e., 
the spherical function 92 a U extends holomorphically to Ac. Simply consider 
G = S1(2,C). Then q = so(3, 1), which in our previous notation corresponds 
to the case p = 2 and q = 0. The explicit formula for spherical functions on 
complex groups specialized to Sl(2, C) then reads 

<p ( e *) = _£ ! . 

^ M ; A e z -e~ z 

Hence we see that TA imax = b c for A g" Z while for A G Z one has TA )max = ac- 



The proof of Theorem 5.1 is computational and is presented in lemmas 
for the various cases. 



670 



BERNHARD KROTZ AND ROBERT J. STANTON 



With our parametrization it follows from (5.1) and Theorem 4.2(iii) that 
for all A G ia*, <p G 



Jrp it 



dX dY 



Ri [(l+ce^||A|| 2 ) 2 +4ce 2 ^||y|| 2 ]^I 7 [(l+ce-^||X|| 2 ) 2 +4ce- 2 ^||Yi| 2 ] — ^ + 7 

Using polar coordinates we thus obtain 

u p ~ 1 v q ~ 1 du dv 

o Jo [(1 + e^u 2 ) 2 + e 2il Pv 2 } E± £ 3 '~^ [(1 + e~^u 2 ) 2 + e~ 2i fv 2 } 2± ^' + ^ 

for a constant c depending on only p and q. Finally with the substitution 
r = u 2 , s = v 2 we arrive at 

(5.2) ifx(e^) = Ce- iX ^ 

p-2 q-2 



OO /"OO 



r 2 s 2 dr ds 



o [(1 + e^r) 2 + e 2 ^s] E ¥ 2 ~' 2i [(l + e -^ r )2 + e -n^ s ^ ± r L +\ 

for all A G ia*, (p G and a constant C which is independent of A. We 
distinguish three cases. 

Case 1: p = 1, q = 0. In this case we have G = Sl(2, M) , the root system 
is split (i.e. A = £), and so u> = u\ = \a. Hence 

a° c = b c = {z£ C:|Imz| < vr} 

and (5.2) boils down to (VA G zR)(V - 7r < <p < ir), 

dr 



(5.3) Me" i n = Ce- iX * 

Jo 



yfr(l + e^r) 2_A (l + e~ 



Lemma 5.3. For p = 1, q = and A G ia*, TA imax = &c = - Moreover, 
for a fixed A i/ie asymptotics at the boundary are given by 

^ A (e-^- £ ))x|log£| 

for e ->■ 0, e > 0. 

Proof. This is immediate from (5.3). □ 

Case 2: p > 1, q = 0. Here g = so(p+ 1, 1) is a classical Lie algebra and 
so fig = b°. We have u = u>\ = a and so o° = {z G C: | Imz| < ^} and 
b° = 2a° . Formula (5.2) simplifies to 

p-2 

(5-4) ^(e - **') = e~ lXip \ p-^r-^ : ?Tx 

Jo (l + e ^r)l^ A (l + e^r)f +A 

for all A G iR and —it < ip < ir. 
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Lemma 5.4. For p > 1, q = and for all A G m*, T A)IIiax = b^ = 2a^ = 
{z G C: | Imz| < 7r}. Moreover, for a fixed A £/ie asymptotics at the boundary 
are given by 

^(e- ?(7r " £) ) - 

for e — »■ 0, e > 0. 

Proof. We will estimate ¥>A( e_ ^' r-e ^) for e — > (e > 0). Since the un- 
bounded contribution to the integral is local (at r = 1) we may henceforth 
assume that A = 0. Then (5.4) gives 



-i(w-e)\ _ 



Me- t(7T - £> ) 



r 2 dr 



|(1 +e i ^- £ )r)\P 



2 e=1 A 

z r 2 dr 

|1 + (-1 + ie)r|P 
r 2 ctr 



o 



o (|l-r| + re)P 



I 
I 



p-2 

i (r + 1) 2 dr 
i (\r\ + (r + l)e)P 

\ dr 
t (\r\+e)P 

-(p-i) 



In the calculation above we used the first order approximation e l( - w £ ) « — 1+ie 
for e > 0, e — > which, as one easily convinces oneself, is justified. □ 

Case 3: p > 1, q > 0. In this case we have u> = uj\ = 2a and so a° = 
{z G C:|Imz| < -|}. Formula (5.2) and Theorem 4.2(iii) then imply for all 

A G iR, -f < </? < f and t > that 



(5.5) (^(i-V^) = Ct^e-*** 



p-2 q-2 

r 2 s 2 dr ds 



roo roo 

J J [(l +t 2 e ^ r )2 +t 4 e 2^ s ]^-|[( 1+e -^ r )2 + e -2^ s ] E ^ 1 + | 

In particular (5.5) implies that (/?^olog^ extends to a holomorphic function 
on b° = 2a° . Again, this turns out to be the maximal domain, as we will show 
below. 

Lemma 5.5. For p > 1, q > and for all A G ia*, T Aimax = b^ = 2a^ = 
{z G C: | Imz| < ^}. Moreover, for a fixed A i/ie asymptotics at the boundary 
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are given by 

^rr ifq>l, 



|loge| ifq = l, 



for e -> 0, e > 0. 



Proof. We will estimate ifi\(e l ^~ £ ^) for e — ► 0, e > 0. Since the un- 
bounded contribution of the integral is local (near r = and s = 1), we may 
henceforth assume that A = 0. Then (5.5) gives that 



p-2 qr-2 

r 2 s 2 (is dr 



P+2<; 



Jo |(1 + e i (|- £ ) r )2 + e i(7T-2e) s ^ 



rco re 




p-2 q — 2 

r 2 s 2 ds dr 



P+2q 



|1 -|- 2re i ^~ £ ) + e i(7r-2 £ ) r 2 _|_ e i(7r-2e) s | £ -2 



■i -2 ^ ^ , , 

z r 2 s 2 ds dr 



p+2q 

lo Jo |i + 2r(z + e) + (-1 + i2e)r 2 + (-1 + i2e)s| — 

.1 .o E^l 9^2 

r 2 g 2 ds dr 
/o Jo ((I + 2re -r 2 - s) + i2(r + e(r 2 + s))] 2 ^ 

.1 „o p^2 9^2 

'o Jo 



r 2 s 2 ds dr 



|1 + 2er - r 2 - s| + 2|e(r 2 + s) + r\ 



P + 2q 
2 



1 -i £^2 £-2 

2 /" r 2 (s + 1) 2 ds dr 

P+2q 

1 (|2er-r 2 -s| +2e(r 2 + s + l) + 2r) 2 



-2 



r 2 ds dr 



~ X (J2er - r 2 - s| + 2e(r 2 + s + 1) + 2rj 
Elimination of the absolute value in the integrand gives 



P+2q ■ 
2 



~2er-r 2 r P 2 ds dr 



p+2q 

' " ! (2er - r 2 - s + 2e(r 2 + s + 1) + 2r) 2 

2 



1 -. 2 

2 r 2 ds dr 



p+2q 

° " (s-2er + r 2 + 2e(r 2 + s + l) + 2r) 2 

p-2 

r 2 ds 



1 (2er-r 2 + 2e(r 2 + l) + 2r + s(-l + 2e 



P+2q 
2 



+ 



1 p-2 

- - 1 r 2 ds dr 



^ p+2g 

° 2: r ~ r ( - 2er + r 2 + 2e(r 2 + 1) + 2r + s(l + 2e)) 2 
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1 E=l , 

r 2 r 2 dr 



s: 



(2e + 2r + 4er 2 ) Ed 2 22 1 

1 £^2 

2 r 2 dr 

~i \ p+ 2 i i 
[r + e) 



2 

1 £^2 , 

_p+2£ +1 /" 2 r 2 dr 



2 " l " ± £ 2 



1 £^2 

£+23 1 1 p-2 /■ 2e r 2 dr 

£ 2 £ 2 £ I 

Jo + 



r — 

q+1 j 2 - 


r 2 dr 


Jo 


(r + 1)— — 1 


r — 

q+1 / 2e 


r~ q dr 


1 

1 loge| 


if Q > 1, 
if 9 = 1. 



□ 



We remark that in order to obtain upper estimates only, the assumption 
that A G io* was not used in view of the degree of generality of the formula in 
Theorem 4.2(iii). Collecting the preceding results we have proved Theorem 5.1. 

Everything that we will have proved about radial limits, namely Theo- 
rem 5.1 and Theorem 4.5, is consistent with the following conjecture. 

Conjecture B. Let a 6 So and H a G o be the corresponding co-root; i.e., 
H a G [9 a ,9 _a ] n a such that a(H a ) = 2. Let c a G R such that c a H a G db°. 
Further, set m a = dimg" for all a G X. Then for all a G So and A G a£. we 
have 

if m a > 1, 



|</? A (exp(i(l - e)c a # a ))| 




if m a = 1. 



Remark 5.6. Correspondence with G. Heckman and E. Opdam suggests 
that the nature of the singularity of the holomorphically extended spherical 
function in co-root directions might be obtained from properties of the mon- 
odromy associated to solutions of the system of invariant differential operators. 

Lower estimates. In a later application to automorphic functions we will 
also need lower estimates for the norm of the if -fixed vector in the holomor- 
phically continued region, for all e > 0, not only at the singularity. The result 
is obtained in a way similar to the preceding. 
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Proposition 5.7. Let (ttx,Hx) be a unitary spherical principal series 
representation of a group G of real rank one. Let X £ db 1 and set 

(v ) £ = 7r A (exp(i(l - e)X))v . 

Then there exists a constant C independent of A such that 

( e (7r— 7e)|A| f ora = n 

\\(v ) £ f = |^(exp(-2i(l - e)X))\ > C | ^ (| _ 21e)|A| J£ \ > \ 

for all < e < 1. 

Proof. As usual we restrict ourselves to the case of A imaginary. 
Case 1: q = 0. Here we have that 



P-2 

, 1 1 2 ■ - — - 1 - ■• '• ' - — ■ '< I 1 2 



i p x (e^) = e-^^ — 

Jo (1 



_|_ e *(7r-e) r ^ 2 -A|2 

By the Weyl group invariance of <px we have fx = (f-\ and so we may 
assume that A 6 iR + , i.e., A = i\X\. Then we get 

IK ° j£ " " Jo |(1 + e i( 7 r-) r )§-A|2 ■ 

If 2 is a complex number, then we write —it < arg(z) < tt for the argument 
of z and m(z) for the modulus of z. Then for < r < \ we have < 
arg(l + e i(7r_£) r) < 3e and m(l + e i(7r ~ £ V) < 2. Hence 

\ > 2~le- fe l A l 

|(! +e i(7r- £ ) r )f-A|2 - 

and the assertion of the proposition for q = follows. 
Case 2: q > 0. Here we have that 
||M £ || 2 = V?a(^- £) ) 



J0 |((l +e ^(f-^) r )2 + se i(7r-2 £ )^ 



P+2q A ' 
4 2 12 



By the Weyl group invariance of (f \ we may assume that A 6 iK + and hence 
get 



(vo 



) e ||2> e (f- e )|A| H f 
Jo Jo 



I p-2 p-2 

r 2 s 2 dr ds 



P + 2q X 2 " 



((1 + e^f- £ V) 2 + se^- 22 )) 2 ^-^ 



Now for < r < e and < s < \ we have < arg((l + e^2 £ ) r ) 2 + 
se *(T-2e)) < io e and m((l + e i( i~ £) r) 2 + se^*" -2 ^) < 2. Hence the assertion 
follows as in Case 1. □ 
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6. Invariant seminorms 

Bernstein and Reznikov, in [BeRe99], introduced the notion of a maxi- 
mal invariant seminorm associated to Sobolev norms of vectors in representa- 
tions. For the K-Rxed vector of spherical principal series representations for 
G = S1(2,R) they coupled this with some estimates on the holomorphically 
extended spherical functions into a beautiful technique to get estimates on 
Rankin-Selberg integrals for Maafi forms. 

We shall extend their technique in several directions. First, by using a 
more representation theoretic viewpoint we will be able to treat the case of 
real rank one groups. When specialized to G = S1(2,R) this will allow us to 
get a small improvement over the corresponding results in [BeRe99] . Secondly, 
in Section 9 we are able to consider some higher rank groups for which we 
obtain estimates on triple products of Maafi forms. These higher rank results 
are likely new, but should be viewed as a sample of the technique rather than 
as sharp results. 

Definition 6.1. (cf. [BeRe99, App. A]). 

(a) Let V be a real or complex vector space and (A^)j 6 / a family of semi- 
norms on it. Then 

(inf ie/ A^):=inf E Vi=v ^ N M) 

also defines a seminorm on V and satisfies inf j 6 j N{ < Nj for every j G I. 

(b) Let G be a semigroup acting on V and N: V — > [0, oo[ a single semi- 
norm. Then for g £ G define a seminorm N g by N g (v) = N{g ■ v). As in (a) 
one obtains a seminorm N by setting 

N G = mi geG N g . 

Definition 6.2. Let (ir, H.) be a unitary representation of a Lie group G 
on some Hilbert space H. Let {X u ...X n } be a basis of g. Then the k th 
Sobolev norm on H°° is defined by 

S k (v)= J2 \\dn(Xr...X^)v\\ (v £ H°°). 

0<mi + ...+m n <k 

It is easy to see that a different choice of basis leads to an equivalent seminorm. 
We remark that S k , k > 1, is usually not G-invariant. As in (b) above, we set 

SC(v)=mf geG S k (ir(g)v). 

Then it is a natural problem to estimate S^(-) for the various representa- 
tions of G. Fix an irreducible unitary representation of a semisimple Lie group 
G having a nonzero if-fixed vector vq. Let v £ T~1\k be a If -finite vector. 
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Recall from Proposition 4.1 that the orbit map G — > Tt\, g i— > ir\(g)v extends 
to a holomorphic map on GT2Kc- Write Q = AQi with fij C exp(za), and 
notice that fij has compact closure. We shall show for real rank one groups 
that (vr A (a)v) is comparable to ||7r^(a)t;|| uniformly inaG fi for all K-finite 
vectors. Similar results will be obtained for holomorphic discrete series in Sec- 
tion 8. But first we explain how for spherical principal series the case of an 
arbitrary if-finite vector v can be reduced to the spherical vector vq. 

Reduction to a spherical vector. 

Lemma 6.3. Let G = KAN be any Iwasawa decomposition and set 
L = AN. Suppose that { / K\,'H\) is an irreducible unitary representation of a 
semisimple Lie group G having a nonzero K -fixed vector vq. 

(i) The K-spherical vector vo is L-cyclic, i.e. 7i\ = span c {7TA(L)t>o}- 

(ii) IfH\K denotes the K-finite vectors of (ir\,Tt\), then 

'H\,K = dir x (U(k))vo, 
where I denotes the Lie algebra of L. 

Proof, (i) This follows from it\(L)vo = 7r\(G)vo and the irreducibility of 
(tta, Ha)- 

(ii) This is immediate from (i). □ 

Let (tt, Ti) be a Hilbert representation of G. For a closed subgroup L < G 
write Hf^ for the smooth vectors for tt\l- If vr is irreducible, then from the 
Casselman-Wallach theory of smooth globalizations of Harish-Chandra mod- 
ules (cf. [Wal92, Ch. 11]) one has that = Hf. 

If H < G is a subgroup, denote by S^h the k th Sobolev norm for the 
representation tt\h- In particular, the Frechet topology on Ti 00 is also induced 
by the Sobolev norms (Sk : K)keN- 

Lemma 6.4. Let G = KAN be any Iwasawa decomposition and set 
L = AN. Suppose that (ir\,Tt\) is an irreducible unitary representation of a 
semisimple Lie group G having a nonzero K- fixed vector vq. 

(i) For every k G N there exist an I G N and a constant C > such that 

(Va G Sli) S k (TT X {a)v ) < CSi tL (TTx{a)v ). 

(ii) For every v G H\ t K and k G N there exist an I > k and a constant C > 
such that 

(Va G fii) S k (TT X {a)v) < CSi, L (ir x {a)vo). 
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Proof, (i) We identify U(qc) with S($c)- Then the natural grading of 
<S(g<c) yields a direct sum decomposition U(qc) = (&k£NM(Qc) k - Fix a norm 
|| • || on gc an d take its natural extension to S(qc). For any g G Gc, Ad (g) 
maps U(gc) k to itself boundedly, so has a norm, say, || Ad(<7)||fc. If X G U(Qc) k 
with ||X|| = 1, then 

||A"7r A (a)w || = ||7r A (a)(Ad(a) _1 X)wo|| < || Ad(a _1 )|| fe sup y6M(0c)fc ||7r A (a)Yv ||. 

imi<i 

Here C = sup aG Q. || Ad(a ±:L )||fc is finite by the relative compactness of fij. 
Hence from Lemma 6.3 there exist an I G N and an r > such that 

||A"7r A (a)t;o|| < Csup Yeu{8c)k ||7r A (a)Yv || < Csup zeM([c)i ||7r A (a)Zw ||. 

||y||<l \\Z\\<r 

Now as i is normalized by o, we get that 

||X7r A (a)v || < C ,2 sup zeM([(c)! ||Z7r A (a)w || < C Si :L (7r x (a)v ) 

\\Z\\<r 

for some constant C independent of X. 

(ii) By Lemma 6.3(h) there exists an X G U(lc) such that v = Xvq. Since 
a normalizes [ the assertion follows now from (i). □ 

Throughout this section we shall follow the custom that a constant 'C 
depends on any quantifiers preceding it in the statement. Thus in the previous 
result (ii), 'C depends on 7r A , k, and v but not on a. 

Compressing Sobolev norms. For any choice of positive roots S + we set 
a + = {X G a: (Va G S + ) a(X) > 0} and a~ = — a + , and, on the group side, 
let A ± = exp(o ± ). 

Lemma 6.5. Let (tt,7{) be a unitary representation of G and v G TL 00 . 
Then for k G No, 

(i) S£>) = |M|; 

(ii) S^ AN (v) = S^ A (v). 

Proof, (i) Let {Xi, . . . , X s } be a basis of root vectors of n corresponding 
to roots cci, . . . , a s G S + . Then for any u G 

<W) = H + £ Nvr(Xr--Xr^||. 
l<mi+...+m s <l 

For a = exp(A") £ A, X £ a, 
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Si,N(ir(a)v) 

= ||tt(o)i;||+ ]T WiX™ 1 ■ ■X^')^{a)v\\ 

l<mi+...+m s <l 

= \H + Yl |K(a)d^((Ad(a- 1 )X 1 ) mi • •(Ad(a- 1 )X s ) ms )t;|| 

Kmi+...+m s <i 

= \\v\\+ Yl e-Zj=i miai{X) \\dn(XT 1 --X?')v\\. 

Kmi+...+m s <i 

If we choose X G a + , 

S£n( v ) < inr aeA+ Si iN (n(a)v) < inf t > <Sj,jv(7r(exp(iX))u) 

= inf t>0 + Yl e~ t ^=^ a ^\\dir(X^ 1 ■ -X?')v\\\ 

^ Kmi+...+m s <l ' 

On the other hand, clearly \\v\\ < <S^ + (v). Thus ||u|| = S^(v) completing the 
proof of (i). 

(ii) One has the obvious inequality S k ,AN(v) > S k ,A(v), so that S% AN (v) > 
Sfr A (v). On the other hand, 

S k,AN( v ) < inf g€G S k ,AN{K{g)v) 

< mf / l eA+ S kt AN{n{h)v) 

= S k ,A(v), 

so that S% AN (v) < S^ A (v). □ 

The case ofG = Sl(2, R). Our goal is to estimate S^(Tr(a)vo) for all a G fij. 
In this section we shall present extensive details for G = Sl(2, R) as this will 
be the model for the proof later for rank one groups. Here we will consider 
an irreducible unitary spherical principal series representation (ir\,7i\). The 
complementary series and nonspherical principal series representations can be 
shown similarly. Discrete series however will be obtained rather differently in 
Section 8. 

We identify N with R via the mapping n x i-> x (see Appendix A for 
notation). We are going to work in the noncompact realization of tt\ on 

L 2 (N) = L 2 (R). With g- 1 = ^ ° ^ the action of n x {g) is given by 
(6.1) (Tr x (g)f)(x) = \cx + d\ x - 1 f(" A ' ' " 



,cx + d 

for all / G L 2 (R) and x G R. For this module one has 

= {/ G C°°(R): M A ~7(^) G C° 
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We use a usual basis for the Lie algebra of g: 

Ho")' Hoo)' F= {°°\ 

Then a = RH, n = RE and n = RF. With U = E - F we have i = RU. 
Differentiating (6.1) one obtains the formulas 

d 



(6.2) 


d7r X (H) 


= (A 


(6.3) 


d7r x (E) 




(6.4) 


dnx(F) 


= (1 


(6.5) 


dirx(U) 


= (A 


(6.6) 


dTT X (E + F) 


= (1 



2, d 



2 d 
dx ' 

^ dx ' 

l-A)x-(l-x 2 )-^-. 

dx 

We also define the radial operators by 

(Rjf)(x) ;■'•'—/;;,•; 
and define the radial Sobolev norms by 

Sfc,rad(/)=£ ll^/ll" 

From the action of dirx(H) and i?- 7 it is clear that there exists a constant C > 0, 
depending on A; and A, such that for all / £ <S(R) 

(6.7) ^S k , Tad (f) < S M (/) < CS fc , rad (/). 

As remarked by the referee, in (6.2) and (6.4) the coefficient of the deriva- 
tive term has a zero; consequently Sk(v) cannot be majorized by S, ^jj(v) or 
by Sk^iv) in general. However, we shall show in the next proposition that 
there is such a relationship for the G— invariant Sobolev norms. 

PROPOSITION 6.6. Let G = S1(2,R) and (ir\,H\), A e ia*, be an irre- 
ducible unitary spherical principal series representation. Then for every k 6 No 
there exists a C > such that for v 6 TL^ , 

S£(v) < CS° A {v). 

Proof. The A action on K/M = S l has two fixed points, corresponding 
to the two Bruhat cells. In the noncompact realization N they become the 
origin and the point at infinity. We shall estimate Sf?(f) by using first a cut- 
off function at infinity, n, and an elementary estimate there. Near the origin 
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a dilated cutoff localizes sufficiently high derivatives of / to get an estimate. 
Away from the fixed points, motivated by an argument in [BeRe99] and classi- 
cal Littlewood-Paley theory, we use a family of suitably dilated cutoff functions 
which compress the n derivatives in the definition of G-invariant norm to radial 
derivatives thereby obtaining the desired estimate. 

For j G Z we denote by Ij the set {x G R: 2^'~ 1 < \x\ < 2~ j+1 }. For a 
function ip on R we write ipj(x) = ip(2 3 x). Notice that if ip is supported in Iq 
then ipj is supported in Ij, and 

supp(^j) n suppOj+i) c ±[-^-, jj]. 

We take a smooth, nonnegative function ip supported in I and such that for 
every m G No, 

r o if \ x \ < 2- m -\ 

J2fj( x ) = { 1 if 2" m < \x\ < 1, 
j=o I if 2 < \x\. 

Choose a nonnegative function r G C°°(R) with support in {x G R: 
1 < \x\} such that (r + <p)(x) = 1 for |x| > 1. Finally for each m G N define 
the function r m G C,? (R) by r m = 1 - r - S j=o • Notice that supp r m C 
{x G R: |x| < 2~ m } and r m (x) = 1 for |x| < 2~ m_1 . From the properties of 
the ifj and r it is easy to see that for any I > f, Tm(x) = -2 lm p^ (2 m x). 

Let / G Hf. Since 

1 = r+l-T 

= r + r m + ^ cpj 

3=0 

m 

= T + lf + Tm + ^lfij, 
3=1 

then 

rn 

j'=i 

For any choices oi g,gi, . . . ,g m & G, using the definition of S^?, we get 

m 

(6.8) S^(f) < S k ({r + <p)f) + S k (irx(g)(r m f)) + £ 5 fc (7r A fe)(^7))- 

First we consider the term S k ((T + <p)f). From an examination of formulas 
(6.2)-(6.4) one sees that 5 fc ((r + <£>)/) < C5 feF ((r + y>)/) for all / G W£>. 
(Throughout this proof C will denote a constant depending only on k, r, ip 
and A.) Hence we have 

S k ((r + < CS k:W ((r + ip)f) < CS^U) 
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for all / £ Majorizing this term in (6.8) we get 



(6.9) S£(f) < CS kW (f) + S k ((* x (g)T m f)) + E S k (w x (gj)( Vj f)) 

3=1 

for all / € Ttf. 

Next we specify a good choice of the elements g,gi, . . . ,g m € G. For every 
t > denote by bt the element 

From (6.1) it follows that 

(ir x (b t )f)(x)=t^-Vf(tx) 

for all t > and x 6 R. Take #j = 6 2 -j f° r all 1 < j < m and g = 6 2 -( m +!) • 
Notice that for every m all the ff\(9j)(<Pjf) are supported in [—2,2], as is 
TT\(g)(T m f). For any smooth function h supported in [—2,2] we can conclude 
from the formulas (6.2)-(6.5) that Sk(h) < CSk,N(h)- Using this in (6.9) we 
get 

m 

(6.10) S%(f) < CS kW (f) + CS k>N (x x (g)(r m f)) + C'£S k , N (ir x (g j )(<p j f)) 

3=1 

for all f eHf. 

Estimating Sk,N(^\(g)(T m f)), we use Leibniz on r m f and L°° estimates 
on T $ = -2i m ^i\2 m x). From (6.3) one sees that S KN {h) = Eto\\ h(l) \\- 
Then 

(6.11) 

S k A*x(9){r m f)) = E ||^2-^M(r m / )( 2-(^).)|| 



k 

1=0 



1=0 



(m+il(i-A), 



J 2- (m+1) '^ l _ ^T^- n )(2- (m+1) x)/W( 2 -(^+ l )x; 



<ix 



<El2" 
z=o 

xE 

n=0 



.(12+11(1- A) I 



[/ 


2~(m+l)i [ 


. J|x|<2 





/ — n 



j r a-n)( 2 -(m+l) x ^(n)^-( m +l), 



fix 



k , s I 
2^~ A 



E 



=0 



E 

n=0 



2~ (m+l)i i 



I — n , 



rt n) (y)f n (y) 



dy 
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<EI2 

k 



n=0 



I — n 

k / 



E (m + 12 , 1 
\2~^— x \ > 
1 1 <2 mn ^— ' 

n=0 l=n 



[l-n 

k—n 



2(m+l)l 



Malice 



|y|<^ 



\f {n) (y)\ 2 dy 



2 m 



= V |2 i!! ^ A | 1 V 

I I 2 (m+l)n Zv I n I 2-?' 

i=o \ / 



2' 



< 



n=0 

fc 



|j/|<W 



I— 2™ 



t l/ (n) (y)l 2 % 



l/ (n) (y)l 2 dy 



I f7')ll 



o ^ !2J 



E 

n=0 



|2(m+l)n 



l/ (n) (y)l 2 ^ 



Now k is fixed and each of the at most k derivatives is in L 2 , hence the 
integrals can be made uniformly small. So for each / we can choose an m so 
that the last line above is at most II f II. Then we have 



S^f) < CS k ^(f) + C||/|| +C^S k , N (^(9j)('Pjf)) 



for any / £ Thus from (6.10) we obtain 



k m d l 



(6.12) 5 fc G (/)<C^(/)+C||/||+C5:5:i|^(2-i( 1 - A )^(2^.))||. 



1=0 j=l 



k m 



As in (6.11), using Leibniz on 99/, L°° estimates on <p^ , and majorizing the 
binomial coefficients, we get 

k m , r 

< C EE( / 2-^|/ W (2^x)| 2 dx' 
1=0 j=i VJ/ o 

km,- 

= C EE( / 2- 2 'i/ (0 (*)i 2 ^ 

Z=0 j=l 

A; m / „ 

< 4C, EE( / |s7 (,) (*)i 2 cfe' 

z=o j=i ^ 
< 4CS fc , rad (/) < 4CS M (/), 

where the last inequality follows from (6.7) and again C depends only on r, 
A; and A. Thus we get from (6.12) and (6.13) that 

s^f) < cs ktW (f) + c\\f\\ + cs kA (f) < c\\f\\ + cs kAN u) 

for all f £Hf. Now, 

S k ^ CS k, AN 



and, by Lemma 6.5(h), S k < CS kA as was to be shown. 



□ 
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In G = Sl(2, R) the element 

is in K and is a square root of the Weyl group element. It will turn out that 
fco provides a uniform minimizer for Sobolev norms. 

Theorem 6.7. Let G = S1(2,R) and (ir\,H\), A G ia*, be an irreducible 
unitary spherical principal series representation. Then for every k G No there 
exists a C > 0, depending on k and A, such that for all a G fij 

<Sfc,A(7rA(/;o)7rA(a)t>o) < C||7r A (a)v ||- 

In particular, for all a G fij 

Sk(n\(a)vo) < C||7r A (a)?;o||. 

Proof. In view of Proposition 6.6 the second assertion follows from the 
first one. To prove the first assertion notice that 

(6.14) (ta(W)(x) = |x + 1| a - 1 /(^|) 

for all / G L 2 (M). 

We parametrize with a £ and a" 1 , where 

/ e if d-e) o \ 

° £ " ^ o e -f a-) J 

for < e < 1. Then, in the noncompact realization, 7TA(a £ )fo is of the form 
c(A,e)/ e where 

fe( x ) = ~ ~~ " ~~i r 

and c(A, e) is a constant depending on A and e, and is uniformly bounded in e 
(as can be seen from §5). Notice that the poles of f £ , as e —>■ 0, are at x = ±1. 
Thus if we take a smooth cut-off function r G C£°(R) with, say, r | [—2,2] = 1> 
then 
(6.15) 

5 M (7r A (fco)/ e ) < 5 M (7r A (fco)r/ e )+5 fc (7r A (fco)(l-r)/ e ) < 5 fe , A (7r A (feo)r/ £ ) + C. 

Here C is a positive constant independent of e because, on the support of 
(1 — r), one has ||(1 — r)f £ \\ x ||(1 — r)x _1 ||, with similar results on the norms 
of derivatives. 

With g £ = r/ e , in view of (6.7), (6.14) and (6.15), it suffices to show that 
Sfc, r ad(7r A (£;o)sO < C\\-K\(a £ )vo\\ 
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for all £ and some constant C > 0. By the radial Sobolev norms and the 
estimate in Lemma 5.3, \\g £ \\ x 1 1 7r^ (a e )^o 1 1 ~ \/| log £ | , it is enough to show- 
that 

\\RjK\(ko)ge\\ < Cj \J\loge\ 



for all j > 1. 

For / G C c °° 



and from (6.14), 

(fll7T A (fcd)/)(x) = 2X|X + 1| A - 1 



,( x — 1 

X+l 



(x + l) 2 ' 

+£ (x)(A-l)x|x + l| A - 2 /(^j) 



with e(x) = 1 for x > — 1 and e(x) = —1 for x < —1. Disregarding the sign 
function e(x), and using induction we have 

3 

(Rj-K X {kQ)f){x) = x 3 Y^ c m P m {x) 

for some constants c m independent of /, and where 

' x — 1 



^(x) = |* + l| A ^~"7 {m) ( 



x + 1 



Thus to estimate 5 , fc ira d(vrA(fco)fl , e ) = X^=o ll-^j^A^o)^!! we must show that 



(6.16) 



{x ] dm^ 3 d n )\ < C\\oge\ 



for all m, n < j. 

Now, consider an expression of the form \ {x 3 x 3 f£)\ where 



(6.17) 



fLx J ti)\ < I * 2j \x + l\' 2 ~ m ~ n ^ |/H 

JR \X + 1/ 



X — 1 
X + l 



dx 



x + l 



l-x 



x + l 



l-x 



-m~n—2j 



|/""'(i)/«"'(x)| <fa 



= 2 / 

= 2 - (m +n)-2j+l /" | x + 1 |2j \l_ x \m+n |/("0( x )/(«)(a;)| 
Jl 

Next, as e — > 0, the functions <? e have poles at x = 1 and x = — 1. 
Similarly, as e — > 0, (x) has poles only at x = ±1 and of order at most 
m+\. Examining (6.17) with / = g £ we see that the factor | x + 1 1 ^ \ 1 - x | ( m+n ) | 
cancels poles. In particular 

|x + l| 2j '|l - x| m+n |^ m )(x)^ n )(x)| 

has poles at x = ±1 for e — > of order no more than that of g e . This establishes 
(6.16) and concludes the proof of the theorem. □ 
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Remark 6.8. (a) The second estimate, S^(n\(a)vo) < C\\ir\(a)vo\\, in 
Theorem 6.7 is optimal in the sense that by G-invariance one has ||u|| < S^(v) 
for any smooth vector v in a unitary representation (tt,7{) of G. 

(b) One can modify the proof of Theorem 6.7 to give the more general 
result 

(Va G Oj) S k ,A(K\(k )ir\(a)v) < C\\ir\(a)v\\ 
for an arbitrary ET-finite vector v. 

(c) The estimate S^(TT\(a £ )vo) < Cy/\ loge| in Theorem 6.7 is a little 
sharper than the estimate (0.5) in [BeRe99], viz. S^! (ir\(a £ )vo) < C|loge|. 

(d) Theorem 6.7 can be easily generalized to complementary series using 
the results on spherical functions in Theorem 4.2. 

Part of the method for G = S1(2,R) generalizes to all groups of real rank 
one. For example, the element ko € K can be found in these groups and gives 
a uniform minimizer for Sk,A- 

Lemma 6.9. Let q be a semisimple Lie algebra with Iwasawa decomposi- 
tion q = t © o © n. Suppose that the restricted root system S satisfies one of 
the following assumptions: 

(1) £ is of type A\ or BCi, i.e., q is of real rank one; 

(2) £ is of type C n or BC n for n > 2. 
Then there exists a ko £ K such that 

Ad(fc )a C£©n = £ffin. 

Proof. First recall that all maximal abelian subspaces in p are conjugate 
under Ad (if). 

Suppose that (1) is satisfied. Then a is one-dimensional. Pick a nonzero 
root vector X a £ Q a . Then e = M(X Q — 6(X a )) is a maximal abelian subspace 
in p which lies in £ © n. Hence there exists a ko 6 K such that Ad(/co)a = e. 

Suppose then that (2) is satisfied. Since S is of type C n or BC n we 
can find a maximal set 71, . . . ,j n of long strongly orthogonal roots. But via 
s[(2,M)-reduction, the assertion follows from the already established rank one 
case above. □ 

We shall make the standing assumption, for the rest of this subsection, 
that G has real rank one. We need to make the element ko more explicit. Let (5 
denote the long positive root. Then we have ft = a if q = 0, otherwise (5 = 2a. 
Choose an s[(2, R)-triple {E, F, H} in g such that E lies in the root space g^, 
F = —BE and such that 



H = [E, F] [H, E] = 2E [H, F] = -2F. 
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With U = E — F we choose 

TT 

k = exp(-J7). 

Then 

Ad(A;o)~ 1 i? = E + F. 

Notice that 

SU = {exp(i<pH)\(p €] - -, - [} 
and introduce elements a e by 

TT 

a £ = exp(i-(l - e)H). 

Proposition 6.10. Suppose that G is of real rank one and that (tt,H) 
is an irreducible unitary representation with K-spherical vector vq. Then for 
all k G No there exists a constant C > such that 

k 

(Va G SU) S k , A (<koMa)v ) < |(/ + cT^ Sj(ir(a)vo). 

j=0 

Proof. S k:A is given by 

k 

S kA {v) = Y,\\H j v\\. 

j=0 

We are going to prove the proposition by induction on k. As the case 
k = is obvious, we start with the case k = 1, so that 

Si,a(v) = \\v\\ + \\Hv\\. 

For a £ flj we obtain 

HTT(k )TT(a)v = TT(k )(E + F)TT(a)v = TT(ko)TT(a)(a~ (3 E + a (3 F)v 
= Tr(k )TT(a) ({a? + a-P)F + cT' 3 {E - F) ) v 



= Tr(k )TT(a)(( a P + a-P)F)v 

= 7r(fco)(a- /3 (a /3 + a- /3 )F)vr(aH. 

Using the unitarity of tt we get 

Si,A(n(ko)ir(a)v ) = ||7r(a)uo|| + ||-H7r(fco)7r(a)vo|| 

= ||vr(a)w || + ja' 3 + a~ /3 | ||F7r(a)w ||. 

Since ||F7r(a)t>o|| < CSi(ir(a)vo), the proof of the k = 1 case is complete. 
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Suppose that the statement holds for k — 1. We must show that 

k 

\\H k 7r(k )ir(a)vo\\ < \a p + Sj(ir(a)vo). 

3=0 

As before we have 

H k 7r(k )Tr(a)v = n{k )(E + F) k 7t(a)v = Tr(k )Tr{a)(a- /3 E + a p F) k v . 

Now we must arrange the expressions (a~ l3 E + a@F) k in an appropriate way. 
With U = E-F, 

{a-PE + a?F) k = ({a 13 + a~ p )F + a~^U) k . 

Now using repeatedly the fact 

(VX G U{Q C ) j ) UXv = ([U, X] + XU)v = [U, X]v , 

with [U, X] e W(flc) -7 ' -1 ) we obtain elements Z Jia G W(0c) J , uniformly bounded 
depending on a, such that 

k 

(a-PE + a^F^o = + a-PyZ jta vo 

3=0 

just as in the = 1 case. □ 

In (6.6) one can see that the derivative term of E + F has coefficient 
vanishing precisely at x = ±1, the eventual poles of f 6 . This is the key to the 
proof of Theorem 6.7. For the other rank one groups the singularity of f £ lies 
on a hypersurface, such as ||x|| = ±1, and this cannot be dominated by a single 
operator E + F, nevertheless an argument of this type will be developed. 

N vector fields from principal series representations. Before we can com- 
plete our discussions for the rank one case, we first have to provide some simple 
facts related to the Bruhat decomposition. The facts collected below hold for 
an arbitrary semisimple Lie group. 

For every k € K we write X k :K/M -> K/M, xM i-> kxM for the left 
translation on K/M. There is the standard action of G on K/M by 

G x K/M -► K/M, (g, kM) ^ n{gk)M. 

Then every leg defines a vector field X on K/M via 

n(exjp(tX)m) 



x - d 

Xm ~Jt 



t=o 



for all m € M. Write p%: g — > 6 for the projection along a+n and pj/ m : — > t/ m 
for the composition of and the quotient mapping 6 — > £/m. 



688 



BERNHARD KROTZ AND ROBERT J. STANTON 



Lemma 6.11. For all X £ g and m = kM £ K/M, 
X m = d\ k (l)p Vm (Aal(k- l )X). 



Proof. By definition we have that 
d 



Xm ~ dt 



n(exp(tX)k)M = ^~ 



/e(lfeexp(tAd(Ar 1 ).X'))M. 



t=o 

Set Y = Ad(k~ 1 )X. Then there exist smooth curves Y t (t) £ t, Y a (t) £ a, 
Y n (t) £ n such that 

exp(ty) = exp(y e (t))exp(y a (t))exp(y n (t)) 

for all tel. Differentiation at t = yields 

y = ^'(0) + ^(0) + ^(0) 

and so Y£(0) = p%(Y). Hence we get that 
d 



X m 



dt 

d\ k (l 



n(k exp(Y t (t)) exp(Y a (t)) exp(Y n (t)))M 

t=o 

K(exp(y { (t)))M 

t=Q 



dt 

= dA fe (l) Pe/m (Ad(fc- 1 )X). n 



The Bruhat decomposition of G 

G= \J NwMAN 

gives the familiar decomposition of the flag manifold into Schubert cells 

K/M = (J k(NwM)/M. 

w<=W 

Lemma 6.12. For all m £ k(N)M/M C K/M, 
T m K/M = {X m :X en}. 

Proof. This result is known but we include it for completeness. In view 
of Lemma 6.11, it suffices to show that for fc = nman, n £ N ,m £ M, a £ A, 

n£ N, 

p e (Ad(fc -1 )n) +m = t 

or equivalently 

Ad(/c _1 )n + m + a + n = g. 
By the special choice of k, 
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Ad(k 1 )n + m + a + n = Ad(raan) 1 ^Ad(n x )n + Ad(man)(m + a + n)) 

= Ad(man)" 1 (n + m + a + n) = g. □ 

Lemma 6.13. Let (ir\,7i.\) be a spherical principal series representation 
of G realized in L 2 (K/M). Then for all m G k(N)M/M C A/M, 

span c {{d7r A (A) m : A G n} U {1}} D T m (K/M). 

Proof. Recall that the G-action on H\ is given by 

(7r x (g)f)(kM) = f(K{g- l k)M)a(g- l k) x -e. 

Hence we get for all X G that 

d 



(d7T X (X)f)(kM) : (U 



f(K{exp{-tX)k)M)a{exp{-tX)k) 

t=o 



X-p 



-(Xf)(kM) + f(k)(X - p)( Pa (Ad(k- 1 )X 



with p a : q — > a the projection along ^ + n. In view of Lemma 6.12, this 
concludes the proof of the lemma. □ 

Truncation at infinity. Recall that G is a semisimple Lie group of real 
rank one and (irx,7i\), A G ia*, a unitary principal series representation of G 
which is realized on L 2 (N), with A = M p © M 9 . 

For < e < 1 let a £ G 0^ be as before. Define a function on W © by 

fe( - X,Y) = [(l + e^l-^UXll 2 ) 2 + e ^-2 £ )||y||2]£±22-f ' 

Then, in the noncompact realization of 7Ta on L 2 (N), the vector 7TA(a £ )vo 
is of the form c(A,e)/ e where c(A,e) is a constant depending only on A and e 
and uniformly bounded in e (cf. §5). 

In order to estimate Sk(TT\(a £ )vo) first we show that the behaviour at 
infinity does not contribute to the singularity as e — > 0. 

Lemma 6.14. Let (7T\,Tt\), A G ia*, be a unitary spherical principal 
series representation of a semisimple Lie group G of real rank one. Let k G N. 
Then there exists a constant C > depending on A and k such that 

(Va G Qi) S k (TT\{a)v ) < CS^fr^^vo) 

for a function r G C^°(A) with support in B:={ZG N: \\Z\\ < 2}. 

Proof. To begin we use the compact realization of TL\ as L?(KjM). Then 
Ti™ = C°°{K/M) and the topology on Tt^ is induced from the usual Sobolev 
norms (S'fc ) x)fc>o an d, in addition, each is equivalent to Sk : K- 
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The Bruhat decomposition gives 

K/M = k(N)M/M U {w} 

where W a = {1,^}- Recall that 

(ir x (a)vo){k) = a(a- 1 k) x -f. 

Let if £ C°°{K/M) with tp = 1 in a small neighborhood of w. We claim that 
we can make supp if small enough so that 

S k ,K(<pn\(a)vo) < C 

for all agfij. In fact, we have (TT\(a)vo)(w) = e ( x -p)( l °s wa ^ from which our 
claim easily follows. 

In order to estimate Sk(n\(a)vo) we have just seen that we can truncate 
the function ir\(a)vo away from infinity. In particular, we claim that there 
exists a constant C > such that 

S k (Tr\(a)v ) < CS k ^((l - (p)TT X {a)v ) 

for all a £ In fact it follows from Lemma 6.13 that, uniformly for all a £ fij, 

Sk,K{ir\(a)v ) < S kjK ({l - ip)irx(a)vo) + S kjK (ipTTx(a)v ) 
< CS k)J f((l - <p)*x(a)vo) + C. 

Now set r = 1 — if so that r is in C^°(N) and we can arrange the support 
so that suppr CB. □ 

XocoZ estimates for invariant Sobolev norms. As before G denotes a 
semisimple Lie group of real rank one and (tt\, Ti.\), A £ ia*, a unitary principal 
series representation of G realized on L 2 (N). 

Recall that a = RA a and define for t > the elements bt = exp(logtA a ) 
£ A. Then for / £ L 2 (N), 

(n x (b t )f)(X,Y) =t Ed r 1 - x f(tX,t 2 Y) 

for all t > and (X, Y) £ W W. Also note the action of N on L 2 (N): 

(ir x (Y)f)(X) = f(X-Y) 

for all X, Y £ A. We also will use the notation u = \\X\\ and v = \\Y\\ for 
(A,y) £ RP©R9. 

Our goal is now to obtain estimates for S^(f) for functions with support 
in the ball 5 = {Z£.A:||Z|| <2}. Especially we are interested in estimating 
Sk{ T fe)- Now for e —> the singularity of f £ lies on the sphere u = 1 for q = 
while for q > the singularity lies on« = and v = 1. This makes it necessary 
to distinguish the cases g = and g > 0. 
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The case of q = 0. N = W in this case and the singularities of f £ for 
e — > lie on the sphere u = 1. 

Lemma 6.15. Suppose that q = and let B = {X £ W: \\X\\ < 2}. Then 
for every k £ No there exists a constant C > depending on k and A such that 

s%(f)<cJ2 II k-i| l7| 5 7 /|| 

for all smooth functions f with support in B. 

Proof. The proof is very similar to the proof of Proposition 6.6, just more 
technical and with more notation. We present the details for the case p = 2, 
the general case simply having more spherical coordinate variables. 

Let T\, . . . ,r n be smooth nonnegative functions with J2?=i Tj = 1 on B. 
Then for any smooth function / with support in B the definition of the invari- 
ant Sobolev norms implies that 

(6.18) Sg{f)<f2SkM9i)(Tif)) 

i=i 

for any choice of gi, . . . ,g n £ G. 

Recall that for all smooth functions h with support in 1QB we have 

S k (h) < CS k ^(h) 

for a constant C depending only on k and A. If we choose our elements gi such 
that supp(7TA(gi)Tj) C 16B, we get from (6.18) that 

n 

(6-19) S^f) < Cj2S k ,NM9i)(Tif)). 

i=i 

Next we make a good choice of functions n and elements gi. We start 
with the Tj. 

Let <f(x) be the one variable function from the proof of Proposition 6.6. 
For any j G No define a function on MP by 

<p j {X) = <p{V{l-u)) 

and notice that 

(6.20) supp^ C {X £ R p : 2^'" 1 < \u - 1| < 2^' +1 } . 
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From now on we use the fact that p = 2. Then elements X £ IR 2 are 
written in polar coordinates as X = (Mcos27r6',nsin27r6') with 9 G R. Now 
choose a nonnegative smooth function ip(9) with support in [0, 2] such that 

ip(m + •) = 1. 
Fix j > 2. For Z 6 Z we define functions on M by 

Notice that 

(6.21) supply C 

As j > 2 these intervals have length at most ^ and so these functions descend 
to smooth functions on the circle R/Z. In particular, we obtain that 

(=0 

for all € R/Z. 

Now we can define our partition of unity. We fix m > 2 and define for all 
2 < j < m — 1 and < Z < 2 J the functions 

T jtl (x) = <p j (xw j , l {e) 

where X = (ucos2ir9,usm2ir9). 

Recall the one-variable functions r m from Proposition 6.6. We define 
functions r mj /, < / < 2 m by 

T m ,i(^) = T m _l(l - U)tp m - 1; l(9) . 

We claim that we have for all 7 £ Nq with I7I < A; 

(6.22) Halloo < C2l^' 

for a constant C depending only on k, ip and 99. In fact it follows from (6.20) 
that all the Tjj are supported away from the origin where the cartesian partial 
derivatives can be dominated by spherical partial derivatives Jjj and The 
claim follows then from the construction of the T m j. 

It also follows from the construction of the tjj that we have for all choices 
of j, I that 

(6.23) diam(suppTj,i) < 16 • 2~ j . 

For a fixed / as in the statement of the lemma we may assume that 

m 2i 

E E T h i = 1 011 su pp f- 

j=2 1=0 



I 1 + 2- 

27' 2i 
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Then from (6.19) we get 

m 1 j 

(6.24) S^{f) < c^^s^y^/)) 

3=1 1=0 

for elements 6 G to be specified. 

For each j, I we now pick an element Zjj G suppr^;. Then define 

g hl = {-Z hl )b 2 - 3 G AN C G 

and notice that 

(6.25) (^ v )/i)(X) = 2-^+^(2-^ + 

for all functions li and I g M p . Then (6.20), (6.21), (6.23) and (6.25) imply 
that 

We choose m large enough such that 

2 m 

H 5 fc,lv(^A(5mj)T-m,j/) < 11/11 
J"=0 

holds. This is possible in view of (6.22) and (6.25). Also from (6.22) we can 
obtain 

#(/) < 11/11 

m-l 2i , si 

+ C E E E( / 2-^M2-*|(^/)(2-^ + Z J -,)| 2 dX) a 

| 7 |< fc j=0 1=0 \J^(sup P T jtl -Z jtl ) 



m-l li , „ s I 

2 



< ll/ll + C E E E / 2-=WN|(^/)(X)| 2 

| 7 |<fc i=0 /=0 VJsu PP T i,i 7 

Now on supply we have 2~ J < 2|u — 1| by (6.20) and so it follows that 



m-l 2^ , „ \ — 

2 



tn—i 4- , „ s 

(6.26) ^(/)<||/||+C£ EE / |n-l| 2 H|^/)(X)| 2 dX 

| 7 |< fc j= Z=0 ^suppr,-,, / 

<c £ || K-i| l7l ^/ll, 

|7|<* 

as was to be shown. □ 

The case of q > 0. Here we have N = W © W 1 and the singularity of 
f £ lies on the sphere u = and v = 1. The analog of Lemma 6.15 for this 
geometry is 
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Lemma 6.16. Suppose that q > and let B = {Z £ W © R q : \\Z\\ < 2}. 
Then for every k £ No there exists a constant C > depending on k and A 
such that 

(1) s%(f)<cJ2 II M l7| d 7 /ll 

\l\<k 

and 

(2) S%(f)<C ]T II \v-l\ hl dV\\ 

\l\<k 

for all smooth functions f with support in B. 

Proof. The proof is essentially the same as the one of Lemma 6.15 and we 
describe only the necessary modifications. 

The partition of unity is now by the truncators 

r u {X,Y) = <p(2?u)<p(&(l - v))thj,i{d) 

with the spherical variable of W. Next, as done earlier leading to (6.25), 
choose 

g jt l = (-Z j:l )b 2 -j e AN 

with Zj : i G W and in the support of i^(2 2j ; (1 — v))ip2j,i(6)- Then in the last step 
(6.26) of Lemma 6.15 we can interpret 2~ J ' either as u or |1 — v \. Accordingly 
(1) and (2) follow. □ 

Sharp estimates for invariant Sobolev norms for real rank one. 

Theorem 6.17. Let (tt\,H.\), A G ia*, be a unitary spherical principal 
series representation of a semisimple Lie group G of real rank one. Let k £ N. 
Then there exists a constant C > depending on A and k such that 

(Va G fii) S^(7r x (a)v ) < C||7r A (a)«d||. 

Proof. For m,n£No define a function on iV = K ? x K' by 

1 



fr ,n {XjY) 



[(l + e i(f- £ )||X||2)2 +e ^-2 £ )||y||2]^-t 

Notice that f £ = f™. 

In view of Lemma 6.14, it is sufficient to prove that there exists a constant 
C > such that 



(6-27) S^(Tf™)<C\\7r x (a £ )vo)\\ 

for all < e < 1. Here r 6 C°°(N) is a fixed function with support in the ball 
\\Z\\ < 2, Z € N. 
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In order to keep track of what differentiation does to the function ff ,q we 
define a shift operator C by 



r( f m,n,_) fr> n+2 forn>0 

MJe ) ~ S f m+2,0 f()r n = g 



Now for Z G n the elements dir\(Z) are the usual differentiations on 
AT = W x M 9 . Consider the action of dvr A (Z) on r/™'™. When applied to 
r it essentially does not increase the function for e — > 0. Applied to /J*' 9 it 
increases the exponent — ^ by 1 and multiplies with a function P £ (X, Y"), 
uniformly bounded in e, X and Y, hence can be bounded by a constant. Thus, 
for every Z G n and G No we obtain the inequality 

(2) \Z k rf^\ < C\rC k (f™)\ 

for all < e < 1 and a constant C > independent of e. 

We now distinguish the cases q = and g > 0. We start with the q = 
case. In view of (6.27), (6.28) and Lemma 6.15 it is sufficient to show that 

|| T\u-l\ k fP +2k '°\\<C\\Tf £ \\ 

as ||t/ £ || x ||7T\(a £ )i;o||. Now this estimate is proved as in Lemma 5.5. 

Finally the q > case is proved similarly by employing (6.27), (6.28), 
Lemma 6.16 (1) and the proof of Lemma 5.6. □ 

Remark 6.18. Using the reduction results in Lemma 6.3 and Lemma 6.4 
one can prove for an arbitrary if-finite vector that 

(Va€Oi) Sg(irx{a)v)<C\\n x (a)v\\. 

Since we do not need this result in this paper and since the proof, albeit more 
complicated, is fundamentally the same, we omit the details. 

However, calculations in the higher rank case prompt us to state the fol- 
lowing conjecture. 

Conjecture C. Let (tt,H) be an irreducible unitary representation of G 
and v a AT-finite vector. Then there exists a constant C > such that 

(Va G Sli) Sg(ir(a)v) < C||7r(a)u||. 



7. Applications to automorphic forms 

For completeness we recall a few notions from automorphic forms. We 
denote by ||g||, g G G, the operator norm of Adg, and we write Z(q) for the 
center of the universal enveloping algebra U{q) of g. 
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Definition 7.1. (cf. [Bo97, Ch. V]). Let r < G be a discrete subgroup of 
co-finite volume. A smooth function /: G — ► C is called an automorphic form 
for T if the following conditions are satisfied: 

(Autl) / is left T-invariant, i.e., 7(717) = f(g) for all 7 G T, 5 G G. 

(Aut2) / is right A-finite, i.e., span c {/(-fc): A; G A} is a finite dimensional 
subspace in C°°(G). 

(Aut3) / is Z(g)-finite, i.e., Z(g)f is a finite-dimensional subspace of C 00 (G). 

(Aut4) / is of polynomial growth, i.e., there exists an n G N and a C > 



If (-7T, 7i) is a unitary representation of G, then we write Ti.°° for the Frechet 
submodule of smooth vectors. The space of distribution vectors liT 00 is by 
definition the strong antidual of 7i°°. If T < G is a subgroup, then we write 
(H~°°) r for the T-invariants of HT°° . 

The following proposition is well known but, because it is crucial to the 
approach used, we include its short proof. 

Proposition 7.2. Let (tt,TC) be an irreducible unitary representation ofG, 
n £ (7i~°°) r and v G H a K-finite vector. Then the function 



is an automorphic form. 

Proof. Since A-finite vectors are analytic, the function Q v „ is defined. 
As rj is T-invariant, (Autl) follows, while (Aut2) is a consequence of the 
A-finiteness of v. Since (ir, Tt) is irreducible, (Aut3) is a consequence of Schur's 
Lemma. Finally, the fact that 7i°° has moderate growth (cf. [Wal92, 11.5.1]) 
implies (Aut4). □ 

Remark 7.3. There is also a very useful converse to Proposition 7.2, i.e., 
every automorphic form is a generalized matrix coefficient (cf. [Wal92, 11.9.2]). 

Proposition 7.4. Let G be a semisimple Lie group with G C Gq- Let 
r < G be a co-compact subgroup. Let (ttx,Hx), A G ia* , be a unitary spherical 
principal series representation. Let 7] G (W^°°) r define an embedding 



such that 




9 V , V :G^C, g^ (Tr(g)v,ri) =r]{ir(g)v) 



Then for all k > 
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In particular, since \\ ■ ||oo is G-invariant 

(Vu G T~Cx) Halloo < CSg(v). 

Proof. This is the content of [BeRe99, Lemma 3.3 and Prop. B.2]. □ 

Combining Theorem 6.17, Proposition 7.4 and Remark 6.18 we obtain the 
following L°° estimate on automorphic forms. 

Theorem 7.5. Let (tt\,H\), A G ia*, be a unitary spherical principal 
series representation of a semisimple Lie group G of real rank one. Let r] G 
(H\°°) r ■ For any K-finite vector v G H\k there is a constant C such that for 
all a G Qi 

Pw x (a)v, v \\oo < C\\ir x (a)v\\. 

Triple products of Maafi forms for real rank one. Let T be a co-compact 
discrete subgroup of G and set Y = T\G and X = T\G/K. For A G we 
have defined the if-spherical principal series representation (ir\,12\). If tt\ is 
unitarizable, then 1C\ denotes the Hilbert completion of T>\ in the compact 
realization. Denote by G the unitary dual of G and by G s C G the subset 
corresponding to the i^-spherical representations, i.e., corresponding to the 
unitarizable i^-spherical principal series. It is then convenient to consider G s 
as a subset of a* c by identifying the equivalence class of tt\ with A. 

For r < G co-compact the Plancherel theorem for the right regular action 
of G on L 2 (T\G) says 

(7.1) L 2 (r\G)-0^ g m^. 

Here m n = dim(/C~°°) r < oo is the multiplicity of (tt,^) in L 2 (T\G). If 
7] G (/C~°°) r , then the G-equivariant map 

K%>^C°°(r\G), v~(rg~(n(g)v,r,)) 

extends (up to multiplication by a scalar) to an isometry K,^ — > L 2 (r\G). 

Write K,^ for the subspace of K- fixed elements and recall that dim K,^ = 1 
for tt G G s and zero otherwise. Taking i^-fixed vectors in (7.1) we obtain that 

(7.2) L 2 (r\G/K)-0^gm^f. 

We will identify L 2 (T\G/K) with a subspace of L 2 (r\G). 
If v G /Cf and r] G (/C-°°) r , then 

ip V(hV (TgK) = (ir(g)v ,r)} 

defines an element in C°°(T\G/K). The function i/) V0:V is referred to as a Maafi 
form. 
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Let (V'n)neN be an orthonormal basis of Maafi forms of L 2 (T\G/K). Then 
each ip n (FgK) is of the form (^\ n (g)vQ , rf) for a specific \ n G G s , a unit vector 
vft G /Cf n and a specific n G (H^°°) r . 

Fix Xo G 90j and for v G Ha,_r: and < e < 1 set 

v £ = 7TA(exp(«(l - e)X ))u. 

If ^ = 6 VOtV is a MaaB form, then we write ip £ = 8( Vo ) e ,r) G £ 2 (Y) for all 
< e < 1. Now, since ^ is continuous, we have ^ 2 G £ 2 (^0 C L 2 (Y). Hence 
we get 

(7.3) V 2 = ^Citpi with q = (ip 2 ,il>i). 

If one considers (7.3) as an identity in L 2 (Y), then analytic continuation yields 

iei 

for all < e < 1. Taking norms, we get 

(7.4) ii^ 2 ii 2 = Ei^i 2 ii^ii 2 = Ei^i 2 iiW) £ |i 2 - 

iei iei 

Theorem 7.6. Let G be a simple Lie group of real rank one and V < G 
a co- compact discrete subgroup. Then for every Maafi form ip, the coefficients 
Ci of the Fourier series of -ip 2 = J2iei c i^i satisfy the following estimates. 

(i) If q = 0, then there exists a constant C > such that for all T > 1, 

y ic i y« < c { T2 '~ 2 ,fp>h 

(ii) If q > 0, then there exists a constant C > such that for all T > 1, 
^ |Ci| 6 - 1 T(logT) 2 ifq=l. 



|A;|<T 



Proof. We start the proof with the identity (7.4): 



^ll 2 = £M 2 HKy 2 - 



NowH^l^ll^llLll^l^llVellLllM.II 2 . 

(i) If q = 0, then we have by Theorem 7.5 and Theorem 5.1(h) for e — > + , 

„ £ 2_2P ifp>l 



< C 



loge| 2 if p = 1 
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for some constant C. On the other hand from Proposition 5.7 we get a lower 
bound. Thus 

f^j - \ |loge| 2 ifp = 1. 

Setting e = ij, and collecting the \ with |Aj| < T, the assertion in (i) 
follows. 

(ii) By Theorem 7.5, Theorem 5.1(h) and Proposition 5.7 the proof goes 
as in (i). □ 

Remark 7.7. (a) For q = and p = 1 the estimate in Theorem 7.6 (i) is a 
slight improvement of that obtained by Bernstein and Reznikov (cf. [BeRe99]), 
viz. (logT) 3 compared to our (logT) 2 . 

(b) For q = and p = 2 (this corresponds to G/K = H 3 ), Sarnak proved 
in [Sa94] that 

M<C(|A,| 2 + l)ie-fM 
for all i. Our estimate in Theorem 7.5(i) yields the slight improvement to 

\ci\<C\Xi\e-^. 

(c) With a more detailed analysis one can improve on the lower estimate 
in Proposition 5.7 in the case of q = 0. One can show that 

IIM.II 2 = |^(exp(-2*(l - e)X))\ > Ce^W j [ ^ ^ ^ = J; 

for all < e < 1. In particular, this gives a small improvement on the estimates 
of the triple products. 

(d) We have presented these techniques for co-compact T; however, they 
apply equally well to finite volume but not co-compact lattices. We illustrate 
this in the next section where we obtain estimates on triple products of cusp 
forms. 



8. G = S1(2,M) 

Analytic continuation of the discrete series. Let G = Sl(2, R) and choose 
K = SO(2,R) as a maximal compact subgroup. For every m G Z define a 
character Xm of K by setting 

Jf cosO sin0\A ime 
V \ — sm cos 6* / / 

We will identify K with Z by means of the above isomorphism. 
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For every k G N there exists a unitary highest weight representation of G 
with highest weight —k and i^-weight spectrum —k, —k — 2,—k — 4, — For 
k > 2 we obtain the discrete series which can be realized in the holomorphic 
functions on the upper half-plane X = {z G C:Imz > 0}. More precisely, a 
unitary highest weight representation (iTk^k) with highest weight —k, k > 2, 
is given by the Hilbert space 

H k = {/ € O(X): j |/(z)| 2 ^<oo} 
Jh y A K 

and the action 

w)/)w-(«. + «-/(^-i) (.--(: :)). 

We recall from Example 4.3 the notation A, Ac, and Aq. For e > small 
we define elements a £ G ^ by 

/ e i5(i-*) o \ 
° £ " ^ e-iM J • 

If w is a i^T- weight vector of (Tr k ,H k ), then we are interested in estimating 
||7Tfe(a e )v || 2 for e — ► 0. The estimates given thus far have been related to 
principal series representations. We remark that the method we shall follow 
applies more generally to unitary highest weight modules of other groups. 

We have found that estimates for ||7Tfc(a e )i;|| 2 are obtained more easily if 
we switch to the realization on the positive real axis. For k G N we define the 
Hilbert space: 

W k = L 2 (R + ,x k -) = {f:R+^C: f° |/(*)| V — < oo}. 

x Jo x 

Then for k > 2 the mapping 

$ k :W k ^H k , f~* k (f); Mf)(*)= / e txz f{x)x k - 

Jo x 

is, by the Paley- Wiener theorem, up to multiplication by a scalar, an isomor- 
phism of Hilbert spaces. Thus we can transport the G-action on 7i k and ob- 
tain a unitary G-action on W k . Call this representation (p k ,W k ). Of course, 
one also has a unitary highest weight representation (pi,Wi) with highest 
weight —1. 

We have for all k > 1 

(Va > 0)(V/ G W fc ) (p fc (( J a ° 1 ))/)(*) = a fc /(aM- 

For every m £ Z let W™ = {«£ W fe : (Vfc G if) vr fc (fc)v = x m (^)t)}. Then 

- — -oo 
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is the orthogonal decomposition of V\4 into .fT-isotypical components. More- 
over, one can show that 

=C{e- x Pj (x)} 

with Pj(x) a polynomial of degree j. 

For the discrete series of S1(2,R) we come now to their analog of Theo- 
rem 5.1. 

Theorem 8.1. Let (pk,Ti-k) be a unitary highest weight representation 
of G = Sl(2, R) with lowest weight —k £ — N. Then for every K-finite vector 
v £ Wk with weight m £ —k — 2No, 

\\p k (a E )v\\ 2 ~e-W. 

Proof. Let m = —k — 2j. Then v is a multiple of f{x) = e~ x pj(x). Hence 
it suffices to show that for g(x) = e~ x x n one has 

\\p k (a e )g\\ 2 ^e- 2n - k 

for every n £ No- 

Clearly for fixed g the map 

F:Al - W k ; F(a)(x) =a k g(a 2 x) 

is analytic. Since F\a = PfeUOff) we have that 

{Pk{a)g){x) = a k g(a 2 x) 



for all a £ A^. Therefore 



roc „ 

\\p(a £ )g\\ 2 = / \e- a ^(a 2 £ xT\ 2 x k 
Jo 

/ 

JO 



dx 

x 



An I' -2Re(ag)x 2n+fc ^ X 



= \a e \ I e v e ' X 



X 



(Rea 2 y 2n - k = {Ree i ^ £) )- 2n - k x £ - 2n - k . D 



Automorphic forms associated to the discrete series. Let r < G be a 
lattice in G but not co-compact. Let P < G be a parabolic subgroup of G and 
P = MApNp with M = {±7} its Langlands decomposition. Call P cuspidal 

for r if rniVp + {i}. 

Definition 8.2. An automorphic form /: G — > C is called a cusp form if 
for all cuspidal parabolic subgroups P < G, 

(V 5 £ G) [ f(ng) dn = 0. 

J(N P nr)\Np 
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Recall from [Bo97, Cor. 8.7] that every cusp form f:G^C belongs to 
LP(T\G) for all 1 < p < oo. 

If (tt,H) is an irreducible unitary representation of G, with space of 
JT- finite vectors Hk, set 

(n~°°)l = {rj G (H-°°f: 6 v , n is a cusp form for all v G Hk}- 

Note that if v G %, v / 0, then the irreducibility of (tt,H) implies that 

= {n G (ft -00 ) 1 ^,,, is a cusp form}. 

Let now n G (W -00 )^ . Then the map 

H K ^L 2 (T\G), v^e v , v 

gives rise (up to scalar multiple) to an isometric embedding 

n -»• L 2 (r\G). 

For f a if-weight vector, as usual, we set v £ = 7r(a £ )v for all e > small. 

Theorem 8.3. Let (^,7^^), k > 2, be a holomorphic discrete series 
representation. Let r\ G {H^°°)^ a^c? t> G a K-weight vector of weight m. 
Then for e small the following assertions hold: 

t-\ \\a ii - M 

W Wv e ,T)\\L*{T\G) ~ £ 2 ; 

I m\ 

(ii) 1 1 0„ ejr) I |oo < Ce~~ for a constant C depending only on v. 

Proof, (i) is just a restatement of Theorem 8.1, since the embedding Hk — ► 
L 2 (T\G) is (up to scalar multiple) isometric, see Proposition 7.4. 

(ii) First we need a little notation. Let Xi, X2, X3 be a basis of g. The n th 
Sobolev norm of the representation (jtkiHk) is given by the equivalent norm 

(W G Kfe°) S» = ||(1 - f> fe pQ) 2 )^||. 

i=l 

It follows from [BeRe99, Prop. 4.1] that 

\K, v \\oo <CS 3 {v) for all i; G 
Moreover since || • ||oo is G-invariant we obtain 

\\0v,r,\\oo<CSf(v) for allt> G 

where S^(-) = inf 3e c Sk(iTk{g)-) is the infimum seminorm (cf. Definition 6.1). 
We will work with the realization Wfc on R + of ir^. Then 

^(W( flc ))CC[x,i 
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which can be seen either by direct computation or as a special case of a general 
result on unitary highest weight representations (cf. [KrNeOO]). Set D N = 
(1 - Ya=i dir k (Xi) 2 ) N , N e N . It suffices to show that 

infgeG \\D N ir k (g)v £ \\ < C\\v £ \\ 

for any if-weight vector v and a constant C depending only on v and not on e. 
We do this only for a highest weight vector v; the computation for the other 
-ftT-types is similar. Write D N = J2j ! ,i=o a ji x ~ an d note that v is given by 
the function v(x) = e~ x . For t > 0, G R we define elements b t ,n s £ G by 

, M , / I .s 

Then for all f £ W k 



bt = [ o ^ J ' andns= ^i 



(7r fc (nA)/)(x) = e- <M! ^/(te)- 

Thus, 

N 



j",«=o 



and 



j,j',i,i'=o 



dx 

x 



c j+j' e -2Rc(a 2 e )tx x k 



N 



J2 ajia-TViis + ajt) 1 (is + a 2 e tf 
j,j',i,i'=o 



(2Re(a e 2 ))-^'^" +fc ) / 
Jo 



oo ,7™ 
X 



< C ]T |ia + aM ,+, 'r^") e -^' +fc ). 
j,j',i,i'=o 

Taking i = | we get 



AT 2 



i,Z / =0 e 



2 

Finally for s = — ^ the expression \is + is bounded for all < e < 1 and so 
we see that inf^c \\D N n k (g).v £ \\ < Cs~z , completing the proof of (ii). □ 
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Triple products in the co- compact case. We recall how to relate automor- 
phic forms of weight m to automorphic functions on G/K. An automorphic 
form /: G — ► C is called of weight m G Z if 

(VkeK) f(gk)= X m(k)f(g). 

We identify X with G/K by means of the isomorphism G/K — > X, 

g-fT i — ^ For every 2 G X and g = ^ ^ ^ set ^(5, z) = cz + d and 

recall that fx satisfies the cocycle relation (i(gig2, z) = fi(g\,g2 • z)(i(g2, z). For 
m G Z we set ^ m = )U _m and note that (i m (k,i) = Xm(k) for all k £ K. If 
/: G — ► C is of weight m, then the function 

(8.1) F(gK)=J^(g~i)f(g) 
defines an analytic function on G/K which satisfies 

(V 7 G r) F(gK) = ~iU^g~K)~ 1 F{ 1 gK) 
for all gK G G/K. 

We say that f:G — > C is an anti-holomorphic automorphic form if 
-F: G/.fC — > C is an anti-holomorphic function. In the usual notation we write 
M®(F) for the anti-holomorphic cusp forms on X = G/K of weight k G N. 
If (iTk,'Hk) is a unitary highest weight representation, then we write for a 
normalized highest weight vector. If & Vk ,ri is the function on G/K associated 
to 9v k ,ri v i a (8-1)) then it can be deduced with the help of Proposition 7.2 and 
[Wal92, 11.9.2] that the mapping 

(H^f c ^ M° k (T), v^Ov k , v 

is an isomorphism of (finite dimensional) vector spaces. 

Let /: G — ► C be an automorphic form of weight m. Then |/| factors to a 
function on G/K which we also denote by |/|. Then (8.1) gives 

(8.2) (Vz = x + ty G X) \f\\z) = y' m \F(z)\ 2 . 
Set 

*-(!-')• 

and define p G 0* by p(Xq) = 1. In a slightly different way, we will identify 
with C by means of the isomorphism 

C 1 ^ Oc, z 1 ^ z2p. 

Let (7Tfe,Kfe), /c G No, be a unitary highest weight representation of G 
and 6 VtT) (Tg) = (TTk(g)v,7]) an automorphic form of weight m. Then |#y )7 j| 2 G 
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L 2 (T\G/K) and as in (7.3) we have 

neN 

with 

Cn = {\@v,r]\ j V'n)- 

As before, we are interested in estimating the coefficients c n , the so called triple 
products. For this we use the strategy introduced by Bernstein and Reznikov 
as used in the proof of Theorem 7.6. Of course, here we must employ the 
estimates just obtained related to discrete series. 

Theorem 8.4. Let T be a discrete co-compact subgroup of G = S1(2,M). 
Let V „ be an automorphic form of weight m £ Z associated to a unitary 
highest weight representation (nk,'Hk), k 6 No, of G. If\0 V:V \ 2 = J2 n <=n Cn^n is 
the orthogonal expansion of 1 9 V:V \ 2 in Maafi forms, then there exists a constant 
C > such that for all T > 0, 

|An|<T 

Proof. We start with the identity (7.4), 

1102 || 2 _ V- I |2|| n ||2 

In view of Theorem 8.3, the left-hand side can be estimated as 

WO 2 II 2 < 11(9 II 2 WO W 2 < dr-Hp-H — r< F - 2 H 

for a constant C\ > independent of e > 0. On the other hand, from Propo- 
sition 5.7, 

IK.J 2 > C 2 e^- 7e ^ 
for a constant C2 > independent of n and e. Thus 

Taking e = ^ and collecting all c n with |A n | < T prove the theorem. □ 

Triple products for the noncocompact case. Let P\,...,Pn be a set of 
representatives of the T-conjugacy classes of cuspidal parabolic subgroups. 
Every Pj admits a Levi decomposition Pj = MAjNj. We choose Aj such 
that Lie(A/) is orthogonal to 6 with respect to the Cartan-Killing form on g . 
Write aj-. NjAjK — > Aj for the middle projection in the Iwasawa decomposition 
G = NjAjK. 



706 



BERNHARD KROTZ AND ROBERT J. STANTON 



For every 1 < j < N and s£C with Re s > 1 one defines the Eisenstein 
series by 

^•(r 5 ,s):= Yl e (1+s)p(los ^ (79)) (geG). 
7 e(rnp,)\r 

For convenience we summarize the properties of Eisenstein series needed here, 
as well as the familiar structure of the Plancherel theorem for L 2 (T\G). [Bo97] 
is a convenient reference. The Eisenstein series admit a meromorphic contin- 
uation in the variable s to the entire complex plane (cf. [Bo97, 11.9]). The 
meromorphic continuation of Ej will be also denoted by Ej. We note that 
Ej(-,s), when defined, is an automorphic form in the sense of Definition 7.1 
(cf. [Bo97, Th. 10.4]); Ej(-,s) has no poles on »R (cf. [Bo97, Th. 11.13]). 

We normalize the inner product on )C\ such that the spherical vector 
u o(fiO = a (g)~ X ~ p > 9 £ G, has norm 1. Then the fact that Ej(-,s) is an 
automorphic form together with [Wal92, 11.9.2] implies the existence of an 
r? JiS G (JC~°°) r such that 

(8.3) (a s (g)v s , Vj:S ) = E 3 (Tg,2s) (g G G). 

Let (p, L 2 (T\G)) denote the right regular representation of G on L 2 (r\G). 
We write L 2 (r\G) s for the G- invariant subspace generated by L 2 (T\G/K), i.e., 

L 2 (T\G) S = sp a n{p(G)L 2 (T\G/K)}. 

One has 

(8.4) L 2 (T\G) S = L 2 (T\G) s4 © L 2 (r\G%, c 
where 

(8.5) L 2 (T\G) s4 = ^^ d m^ 
with m w < oo (cf. [Bo97, Th. 16.2, Th. 16.6]), and 

N r(B 

(8.6) L 2 (T\G) S , C = £ / K.j ds 

(cf. [Bo97, Th. 17.7]). In (8.6) the module /C s j is isometrically equivalent to 
JC S and K.fj is realized as the image of the G-equivariant embedding 

(8.7) H? - C°°(r\G), v^(Fg^{a s (g)v, Vj , s )). 

Let (V'n)neN be an orthonormal basis of L 2 (r\G) s L 2 (T\G / K) of Maafi 

cusp forms. Then ip n (Tg) equals {(J\ n {g)vQ,rj) for Vq a normalized if-fixed 
vector in )C\ n and some element i] G (/C^°°) r . 

If / € L 2 (T\G/K), then (8.3)-(8.7) imply that 

oo AT 

(8.8) / = E</'^« + E / (f,E,(;2 iS )}E,(.,2is) ds 

n=l 7 = 1^ 
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and 

oo N 

(8-9) ll/l| 2 = EK/^«)| 2 + E / K/,^(-,2 iS ))| 2 d S . 

n=l j=l JK 

Theorem 8.5. Let (irk,Ttk) be a unitary highest weight representation of 
G and f = 9 V ^ an associated cusp form of weight m. Then \ f\ 2 £ L 2 (T\G/K) 
and there exists a constant C > such that for all T > 0, 

E KI/| 2 ^n)|Vl A "l + E / |(|/| 2 ,^-(-,2z S ))|VNd S <CT 2 H. 

|A„|<T j=l J - T 



Proof. It is clear that |/| is right i^-invariant. Moreover, since cusp forms 
are rapidly decreasing (cf. [Bo97, Th. 7.5]), we have \ f\ 2 G L 2 (T\G/K). Hence 
(8.8) gives 

oo N 

(8.10) |/| 2 = E<I/I 2 '^)^ + E / (l/| 2 ,^(-,2^))^(-,2^)d S . 

„=i ]= i Jk 

We analytically continue (8.10) as in the proof of Theorem 8.4. First notice 
that Ej(-,2s) corresponds via the Plancherel theorem to Vq € /C s , our unit 
if-spherical vector in K, s (cf. (8.3), (8.7)). So we define Ej :£ (-, 2s) as the element 
corresponding to Vq £ . Now analytic continuation of (8.10) in L 2 (r\G) s gives 

oo N r 

(8.11) \f £ \ 2 = E<I/| 2 > V'n^ + E / (\f\ 2 ,E j (.,2is))E j , £ (;i2s)ds. 

n=l j=l jR 

Taking norms in (8.11) we arrive at 

oo N 

(8.12) |||/ £ | 2 || 2 = EKI/| 2 ^n)| 2 K £ ll 2 + E / KI/| 2 ,^(-,2 iS ))| 2 ||^ £ || 2 d S . 

n=l j= l jK 

In the proof of Theorem 8.4 we showed that |||/ £ | 2 || 2 < Ci£~ 2 l m L It follows 
from Proposition 5.7 that there exists a constant C2 > such that ||V(j £ || 2 > 
C 2 e(7r-7 £ )| S | and || v n e ||2 > c 2e (7T-7 £ )|A„| for all s e iR and n G N _ Thug 

(8.13) 

00 jv „ 

EKI/| 2 ^n)| 2 e^ 7£ )l A "l+E / \{\f\\E 3 (.^s))\ 2 e^)\s\ ds <ce- 2 ^. 

n=l j=l jR 

Setting e = y in (8.13) and collecting the appropriate terms prove the 
theorem. □ 
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Remark 8.6. In [Go81a,b] Good proved a special case of Theorem 8.5 
with methods from analytic number theory. To be more specific, for 
k > 2, k even, and / = 9 VkjT) , i.e., \f\ 2 (z) = y k \F(z)\ 2 , z G X, for some 
(anti-)holomorphic automorphic form F on the upper half-plane X, Good 
proved (cf. [Go81a, Th. 1]) the estimate given in Theorem 8.5 for such an /. 
A comparison of the proofs shows the effectiveness of the representation theo- 
retic approach. We should point out that the number theory normalization of 
the Eisenstein series Ej(-,u) used in [Go81a] differs by a change of parameters 
u = 2s — 1 from the representation theory notation used in this paper. 

Estimating Fourier coefficients of holomorphic cusp forms. Let F G M%(T) 
be an anti-holomorphic cusp form on the upper half-plane X. Let P be a cus- 
pidal parabolic subgroup for T. Replacing T with a certain G-conjugate we 
may assume that 

rn7V = {^J ly.nez}. 

Then F is a holomorphic cusp form on X and as such admits a Fourier expan- 
sion at infinity 

oo 

F(z) = ]T c n e 2mnz . 

n=l 

As before we have 

Vz = x + iy€X \f\ 2 (z) = y k \F(z)\ 2 

for some cusp form / = 9 Vk:V associated to a unitary highest weight represen- 
tation (7Tfc, H-k) of G. 

If h G C°°(r\G) is rapidly decreasing, then we define its constant term 

h P :N\G^C, Ng^ [ h(ng) dn. 

J(rnN)\N 

Since |/| 2 is a cusp form and hence rapidly decreasing, the Rankin-Selberg 
convolution theorem (cf. [Bo97, Prop. 10.10]) yields for s G C with Res > 1 
that 

(8.14) (|/| 2 , E(; S )) L2(rv?) = ((|/| 2 )p, e( 1+ ^^<)) L2(iV \ G) 

with E the Eisenstein series associated to P. From (8.14) and a straightforward 
calculation one gets the familiar Rankin-Selberg zeta function (cf. [Bu97, p. 71- 
72]) 

1 i oo 

(8.15) -(47r)^-i T(k - 2 + |) E M 2 n^-f = (|/| 2 , E(; s)) 

n=l 

for all s with Res > 1. From Theorem 8.4 we obtain the estimate 

(8.16) J* \{\f\ 2 ,E(;2is)\ 2 e*\ s \ ds < CT 2k . 
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It was shown by Good in [Go81a, p. 544-547] (see also [Pe95, p. 121-122]) 
that a combination of (8.15) and (8.16) yields the following result: 

Theorem 8.7. Let F(z) = J2n=i c„e 2 ™ 2 be a holomorphic cusp form of 
weight k £ N with respect to an arbitrary discrete subgroup T < G of co- finite 

volume and the property r n iV = { ^ J " j:ii£Z}. Then 

k 1 

|c n | << Tl2"6 +£ 

for every e > 0. 

Fourier coefficients of Maafi forms. We shall give yet another applica- 
tion of holomorphic extension, but with a different technique. Here we view 
Whittaker functions as eigenfunctions of the invariant differential operators on 
the locally symmetric space and use their holomorphic extension. It is classical 
that Whittaker functions have such extensions but this seems to be a new use 
of it. To avoid technicalities we suppose that G = Sl(2, R) and r < G is a 
discrete subgroup with co-finite volume. We assume that V admits at least one 
cusp and that 

mN = {(l ™):n€Z} 



for some c > 0. Let Q v . r] be a Maafi form on T\G/K. For an element z 
x + iy G C with y > we define g z = ^ ^ 
Then 9 V)TI admits a partial Fourier series 



x + iy £ C with y > we define g z = I 1 an< ^ we no * e that 9z'i = z. 



Ov, v (Tg z ) = a Q (y) + ^ a m ^K s (—\m\y)e 2ni ^ x 



where s G C and 

i^(y) = ^ / V^+i)/V ^ (y>0) 

is the if-Bessel function. 

Theorem 8.8. Let 6 V)V be a Maafi cusp form for G = Sl(2, R). Then the 
Fourier coefficients of 6 V;V satisfy 

(ViV>2) ]T — <ClogiV. 

m|<JV ^ 

Proof. For every m £ Z define a unitary character Xm of the circle group 

r n iv\iv by 



1 X 
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dn 



Then we have for all m £ Z, m 7^ 0, the identity 

(8.17) a m ^K s {—\m\y)e 27ri ^ x =f v>r] (ng z )x-m(n) 

c JrnN\N 

for z = x + iy in the upper half-plane. Now we holomorphically extend both 
sides of (8.17). For < e < 1 recall that 



e *i(i-e) 



a F = 





e -<f(l-e) 



e4 



c 



and y e = e^ 1 £ ) and note that = a e . Then by analytic continuation, 

a rn ^/y~ £ K s ( — \m\y e ) = / 6 VtV (na £ )x- m (n) dn 
c JrnN\N 



for all < e < 1. For every e define 



It follows from Theorem 7.5 that H^^Hoo < C\ logeja and so in particular f £ 
is bounded and belongs to L 2 (T n N\N). Therefore we get that 



fe{n) = b mXm{ 



n 



and 



El^| 2 = H/ £ |l2<ll/ £ |lL<C|loge| 



Since b m = a m -Jy^K s (^-\m\y e ) we thus obtain that 



E 



I I Ve 



2tt 2 
K s (—\m\y £ ) < C| log e| 



In particular for all N £ N we get that 

,2tt 



\a m \ 2 K s { — \m\y £ ) <C|loge|. 



m|<JV 



Now choose e = j^. Then for all \m\ < N, 



\m\y £ 



\m\ie n \m\i + 



2\m\ 



and so by the asymptotic expansions of the Bessel functions there exists C > 
such that 



K s ( — \m\y £ 
c 



> 



a 



for \m\ large. This proves the theorem. 



□ 
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Remark 8.9. The Ramanujan conjecture for Maafi forms says that the 
coefficients \a n \ grow more slowly than n £ for all e > 0. Comparison with the 
result in Theorem 8.8 shows that our result is consistent with this conjecture 
and that our result is essentially sharp. A little more care with obtaining an 
asymptotic estimate of C log N, instead of a bound of C log N, together with a 
Tauberian theorem for logarithmic means give the equivalence of such a result 
with the order of the pole of the Rankin-Selberg zeta function. We thank 
Wenzi Luo for an informative conversation on this topic. 



9. G = S1(3,M) 

Now we are going to apply our techniques to a group of higher rank, 
namely G = S1(3,R). This group is low dimensional enough for explicit com- 
putations to be possible, yet it also illustrates that the technique works in 
higher rank as well. In particular, we will verify part of Conjecture B and give 
a complete answer to the boundary behaviour of the analytically continued 
spherical functions in the direction of the extremal rays. Finally, with these 
estimates available we can give an application to triple products. 

Let us briefly summarize the notation for this special case: 

3 

o = {diag(xi,x 2 ,x 3 ):xj G R; Xj = 0} 

i=i 

and 

3 

A = {diag(ai,o 2 ,a 3 ):aj > 0; JJ aj = 1}. 

j'=i 

The positive system is S + = {£i — £2, £i —£3, £2 — £3} and the associated simple 
roots are given by 

II = {£1 - £ 2 , £ 2 - £3}- 

Here uj\ = £1 and u>2 = £1 + £2- The Weyl group, W a = S3, in this case acts 
as the permutation group of {£i,£ 2 ,£3}. 

Define 3x3 matrices E^ by Eij = (5k-ij-j)k,i- Note that Q £i ~ £ i = REij. 
Now ni = q £2 ~ 61 and T12 = £3_£2 © Q 63 ~ £l are subalgebras of n satisfying 
n = tti + n2, with n2 abelian. The map 



M 3 -> N, (x, y, z) i-> exp(xE 2 \) exp(yE 32 ) exp(zE 3 i) 




J 



is a diffeomorphism. We take a choice of Haar measure dn on N so that its 
pullback under $ is the product of Lebesgue measures dx dy dz. 
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Lemma 9.1. For all 

a = diag(ai, a 2 ,a 3 ) G A 

and 

n = exp(x-E 2 i) exp(yE 32 ) exp(zE 3 i) G iV 
the following assertions hold: 

(i) a(an) 2wi =a\ + a\x 2 + a|z 2 ; 

(ii) a(an) 2w2 = a\a\ + a\a\y 2 + a\a\(z + xy) 2 = a^ 2 + a 2 2 y 2 + a{ 2 {z+xy) 2 . 

Proof. This is elementary. □ 
We can improve Theorem 1.8 (ii) slightly. 
Lemma 9.2. For G = Sl(3, R) , 

B£G C ITc^JVc- 

Proof. By the usual argument with the Bruhat decomposition it is enough 
to show that B^N C Ifc-Ajl'-iVc. This follows readily from Lemma 9.1 and 
Lemma 2.1 (i) . □ 

In view of the discussion leading to Theorem 4.2(i), we can say that all 
spherical functions <p\, AG o^, extend to A' l>"J\ . 

Since q is split, p = uj\ + w 2 . Then Theorem 4.2(hi) together with 
Lemma 9.1 gives the following expression for spherical functions on G = 
S1(3,R), valid for all a G expib and all A = \\uji + A 2 w 2 G ia* (the only 
reason we restrict ourselves to A imaginary is to keep the following formula 
manageable) . 
(9-1) 

dx dy dz 



(ai + a 2 x 2 + a^z 2 ) 1 Al (aia 2 + aia 3 y 2 + a 2 a 3 (z + xy) 2 ) 1 x ' 2 

For each a G £ we write H a G o for the co-root of a. Notice that 
\H a G db° for all a G S. 

We consider the radial limits of in the directions of the co-roots and 
the fundamental weights. We begin with the co-root directions. 

LEMMA 9.3. Let a G S and &e f/ie corresponding co-root. Then, for 
all A G for which (tt\,H.\) is unitarizable, 

<p x (exp(i^{l-e)H a )) > C|loge| 

/or a constant C = C(A). 

Proof. This is a special case of Corollary 4.6. □ 
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Next, for the fundamental weights we have 

2 11 112 

X m = diag(-, -3,-3) and x oj 2 = diag(-,-,--) 

and ±irX U}j G db°. 

For these weights we use the following interesting splitting of the spherical 
functions in extremal directions. 

Proposition 9.4. Let G = S1(3,R) and X e RX^ n b°. Put a = 
exp(zX). T/ien, /or a// spherical functions (p\, A G a^, 

¥>A(a) = C/ ' 



;ai + a 2 x 2 + a 3 z 2 ) 1_Al (aia 2 + a 2 a 3 ,z 2 ) 1_A2 
1 



da; dz 



(ai + a 2 x 2 + a 3 2: 2 ) ReAl (aia 2 + a 2 a 3 z 2 ) RcA2 

C = _ a{n')- £2+63 cM' 
Jn' 

and n' = exp(g £3 ~ £2 ). 

Proof. Set b = exp(i|X). Let n' = g £3 ~ £2 and n" = g £2 ~ £l g £3 " £l and 
note that n = n' © n" is a direct sum of subalgebras. Hence iV = N' ' n" = 
N x N with N = exp(n') and N = exp(n"). Hence we get from Theorem 
4.2(iii) that 

ip x (b 2 ) = J_\a(bnf x ~ p) \ • a(on)~ 2RcA dn 

= L L, Hbn'n" f x -ti I • a(bn'n")' 2KcX dn dn" . 
Jn' Jn" 

According to the Iwasawa decomposition, n! = k'a'n'. Since k! commutes 
with 6, 

<P\(b 2 ) = [_ [_ \a{bk'a'n'n") 2 ^-^\ ■ a{bk'a'n'n")- 2ReX dn' dn" 
Jn' Jn" 

= L L \a{ba'n'n"n'- 1 ) 2 ^-^\ ■ a(6a'n'n"(n') _1 )~ 2ReA drf dn". 
Jn' Jn" 

Since N' normalizes N" in a unipotent way, that 

fx(b 2 ) = L [_ \a{bdn") 2{ - x -^\ ■ a(ba'n")- 2KcX dn' dn" 
Jn' Jn" 

= L \a(n') 2 ^ti\a(n'r 2ReX 
Jn' 

x L \a{ba'n"{a'Y l ) 2{ - x ~ p) \ ■ a(ba'n"(a')~ 1 )~ 2RcX dn' dn" 
Jn" 

= L a(n )- 2p [_ \a(ba'n"{a')- 1 ) 2(x -^\ ■ a{ba'n" {a')- 1 )-^^ dn' dn" . 
Jn Jn 
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Now the Jacobian of the map N —>■ N , n" h-> a'n"(a') 1 is given by (a') 7 
with 7 = -((ei - e 2 ) + {e\ - e 3 )). Hence, 

Va(& 2 ) = L a(n')- (£2 ~ £[i) dn' [_ |a(6n") 2(A " p) | • a(on")- 2RcA dn" . 
Jn' Jn" 

Finally Lemma 9.1 gives 

f \a(bn") 2ix - p) \ -a(bn"y 2RcX dn" 
Jn" 

= / 



(ai + a 2 x 2 + a^z 2 ) 1 A i (aia 2 + a 2 a 3 z 2 ) 1 A 2 
1 



dx dz, 



(or + a^x 2 + ap 2 ) Rc A i (alaj + aoM^z 2 ) Re ^ 
concluding the proof of the proposition. □ 

Lemma 9.5. Let A e oc- r/ten /or j = 1, 2, 

|^ A (exp(±i7r(l-e)X Wj .))| <C- 

for a constant C > 0. Furthermore if X £ id*, then 

1 



|p A (exp(±t7r(l-e)X Wj .))| 



Proof. Notice that s ei _ £3 (X tJl ) = —X^ (i.e. for Sl(3, M) 7Ti is contragre- 
dient to 712) so that we have to consider only the radial limits in the direction 
of ±X W1 . We restrict ourselves to the case of —X W1 . Then Proposition 9.4 
gives 

(px(exp(in(l - e)X Ul )) 

1 



C(X) I 

JR- 



|^ e i|w(l-e) +e -iI ff (l-e) a . 2 + e -ii)r(l-e) ;? 2)l-Ai| 

1 



^ e i±ir(l-e) _|_ e -i§7r(l-£) 2 ,2^1-A 2 |^ e -»|^( 1 -£) _)_ e *| 7r ( 1_£ ) 2 ; 2 + e *| 7r ( 1 - £ ) z 2^Rc Ai 
1 



dz. 



^ e -igir(l-e) _|_ e i§7r(l-e) z 2^RcA 2 

Hence we get 

iiir(l-e)(A 2 -Ai)| e i|ir(l-e)(ReA 2 -ReAi) 



¥ . A (exp(i7r(l-e)X a>1 )) = C(A) / 



|( e iT(l-e) +x 2 + ^2)1-Ai| 

1 



|(1 + e -Mr(l-e) z 2)l-Aa|( e -Mr(l-e) + x 2 + 2; 2)ReAi( 1 + e ivr(l- e ) z 2)Re A 2 

We see that the singularity of the integral is located at x 2 + z 2 = 1 and 



z 2 = 1. 
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Let us assume now that A 6 ia*. The upper estimate for general A G o| 
will be proved in the same way. 



So for fixed A G ia* we get 
V 5 A(exp(-ivr(l - e)X LOl )) x 



•2 r 1 



dx dz 



o Jo 

2 r2 



|(eiT(l-e) + x 2 + z 2)( e m(l- £ ) + z 2)| 

dz 

o |(-1 + ii + + z 2 )(-l + ie + z 2 )| 
2 / ' 2 dx dz 

o (e+|l-x 2 -z 2 |)(e + |l-z 2 |) 
2 rl dx dz 

-i (e+\x 2 + z 2 + 2z\)(e + \z\) 
2 rl dx dz 



o Jo (e + x 2 + z 2 + 2z)(e + z) 
dz 



f 

Jo 

f 

Jo 



(e + z 2 + 2z)(e + z) 
dz 



(e + z) 2 
1 



□ 



Putting these results together we have 

Theorem 9.6. Let G = Sl(3, K) and (tt\,7~C\) be a unitarizable principal 
series representation of G. Then for the associated spherical functions: 

(i) If X £ db°, then there exists a constant C > such that 

\<f X (exp(±i(l - e)X))\ < 

(ii) If X = ±7rX UJj G o, j = 1, 2, and A G ia*, then 

|v9 A (exp(i(l-e)X))| x - £ - 

(hi) If X £ db°, then for a constant C > 

|^ A (exp(i(l-e)X))| >C|loge|; 

(iv) For a// A G ia* the domain b^ = a + ib° is i/ie maximal connected tube 
domain a + iuj C ac, w C a, containing suc/i i/tai yj> |a extends holo- 
morphically to exp(a + ia;). 
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Proof. For the proof it is helpful to keep Figure 1 in mind. 

(i) For X an extreme point, the assertion is Lemma 9.5. The general case 
follows from Proposition 4.4 (cf. Figure 1). 

(ii) Lemma 9.5. 

(iii) For a co-root direction this inequality is Lemma 9.3. For X Wj this 
follows from (ii). An arbitrary X G db° is a convex combination of co-roots 
and X w . (cf. Figure 1) so that the result follows from Phragmen-Lindelof as in 
the proof of Proposition 4.4. 

(iv) In view of (iii), this follows from Proposition 4.4 (cf. Figure 1). □ 

Remark 9.7. Theorem 9.6 gives an almost complete description of the 
boundary behaviour for the spherical functions for G = Sl(3, R). Generally we 
expect the following for generic A G o^: 

, , \itxm I - for X G db° extremal 

|^(ex P «l " ^))\ * { fioge| for X G <%° not extremal. 

Lower estimates. For G = S1(3,R) the best lower estimates one can get 
for spherical functions are in the direction of the fundamental weights. We 
illustrate only this case. 

Proposition 9.8. For G = Sl(3, R) and X G b° an extreme point, i.e., 
X = iiTrX ulj , there exists a constant C > such that 

|p A (exp(i(l - e)X))\ > Ce {1 - £)sup ^a wAmX (- iX) 

for all A G d£. 

Proof. We may assume that X = ttX UJi = ir diag(|, — |, — |). In what 
follows we will also see that the assumption A G ia* is justified. Fix < e < 1 
and set a = diag(ai, a2, a%) = exp(i(l — e)X). Recall from (9.1) that 

f dx dy dz 

<P\W ~ 1 



(ai + a 2 x 2 + a^z 2 ) 1 Al (aia 2 + a\a 3 y 2 + a 2 a 3 (z + xy) 2 ) 1 x ' 2 

Hence the fact that a±, a 2 , a 3 are R-collinear in C as well as that ai<2 2 , 0103, a 2 a3 

are R-collinear in C implies that 

1 1 
" 2 



Jo Jo 

dx dy dz 



(ai + a 2 x 2 + a^z 2 ) 1 Al (aia 2 + aia 3 y 2 + a 2 a 3 (z + xy) 2 ) 1 A2 
> Ca Al (aia 2 ) A2 = Ce^^ 1 -^). 
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Now the assertion of the proposition follows from the Weyl group invariance 
of ip\ in the A- variable. □ 

Estimates on automorphic forms. Here T denotes a co-compact discrete 
subgroup of G = S1(3,M) and (ir\,H\) a unitary spherical principal series 
representation of G. Recall from Proposition 7.4 that each i] G (H^°°) r defines 
an embedding 

n oc ^ c°°(T\G), v ^ 6 V , V ; e v jTg) = (7r x (g)v, V ) 

with 

(VveH?) Halloo < CS%(v) 

for k > 4 = \ dim G. As before vq denotes the normalized if-spherical vector 
of (ir\,H\). Fix an extremal element X 6 b 1 , i.e., X = ±^X u)j , and set 

(v ) e = vr A (exp(i(l - e)X))v 

for all < e < 1. As in Section 6 we are interested in the G-invariant Sobolev 
norms S^((vq) £ ) for e — ► 0. 

Proposition 9.9. Let (tt\,H\), A e ia*, 6e a unitary principal series 
representation of G = Sl(3, M) and rj G (H^°°) r /or a co-compact discrete 
subgroup T < G. Then there exists a constant C > suc/i i/ia£ 

for all0<e<l. 

Proof. We may assume that the extreme point X £ b 1 is —\X Wx . Since 
dim G = 8, it follows from Proposition 7.4 that it suffices to estimate ^((do^)- 
We shall show that 

S 5 ((v ) £ ) < 

for all < £ < 1. 

More generally, we will consider Sk((vo) e ) for an arbitrary fc £ Mo. Let 
Xi, ■ ■ ■ , X$ be a basis of 0. Then 

S k (v)= £ \\X^-.-X™*v\\ 

miH hm,8<fc 

for all smooth vectors v. 

We will work with the noncompact realization of tt\, i.e., H\ = L 2 (N). 
Then the vector (vq) £ is of the form c(\,e)F £ , where 

F e (x,y,z) = — 

( e i(7r-e) +x 2 + z 2)^^ + y 2 + e-iCTr-e)^ + xy )2)^ 

and c(A, e) is a constant uniformly bounded in e. 
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We will use results from the proof of Proposition 9.4. We denote the 
change of variables in Proposition 9.4 that yielded the splitting by (x, y, z) ^ 
(p(x,y,z). Recall that <p was given by a composition of two maps (x,y,z) 
(x + yz,y,z) followed by (x,y,z) i-> (g(y)x,y, g(y)~ 1 z) where g(y) = l+y 2 . 
Then 

U:L 2 (N)^L 2 (N), /-/op 
defines a unitary operator. Set 

Mx,y,z) = —— r4 ; : ~, -fl(2/)~ l+(Al ~ A2) • 

( e »(ir-e) + x 2 + z 2 )^~(l + e~ l ^- £ ) z 2 )~^~ 
Then the change of variable formula in Proposition 9.4 implies that 

U(F E )(x,y,z) = f e (x,y,z). 

Define a differential operator Uj = Uo Xj o U^ 1 for j = 1, ■ ■ ■ , 8. For every 
A; we define a seminorm for functions / G C 00 ^ 3 ) by 

W)= E ll^r---^ 8 /!!, 

whenever N k (f) < oo. In particular, we have 

(9.2) S k (F £ ) = N k (f £ ) 

Let B = [-2,2] 3 and let r G C^QR 3 ) with r| B = 1. We claim that 

(9.3) W fc (/ e ) < N k (rf £ ) + C 

for all e > and a constant C > independent of e. Write 

/(*> V, z) = • g(y)-^ +(Xl - X2) 

(1 +x 2 + Z 2 )^~{1 + z 2 )~^~ 

and note that / corresponds to the vector vq after the change of variables. 
Clearly we have 

N k (f) = S k (v ) < oo. 

Now, outside the ball B we can compare f £ with / and our claim (9.3) follows. 
Now define the two variable function g £ (x, z) by 

i 

9e(x,z) 



( e i(T-e) +x 2 + z 2)^- L (' 1+e -i(7r- e ) ;Z 2)^ 

Let T2 G C C (IR 2 ) be such that T2 |[-2,2] 2 = 1- Then it follows from (9.2) and 
(9.3) that there exists a constant C > such that 

S k ((v ) £ ) < C + C £ \\^j^;T 2 g £ \\ L2 

j+l<k 
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The same computation as in Lemma 9.5 then leads to 

— 1 +' 2k 

S k ({vo)e)~£ 2 • 

Specializing to k = 5 proves the proposition. □ 

Remark 9.10. (a) The estimate in Proposition 9.9 is certainly not opti- 
mal. The conjectured optimal estimate would be 

||0(u o )e,r)lloo < C||K) £ || x -j=- 

The major technical difficulty arises from the singularities of the spherical 
vector (vo) e for e — > considered as a function on N. In the rank one case 
the singular locus always is a compact variety, whereas in this case the singular 
locus lies on a complicated unbounded variety. This is reminiscent of the theory 
of intertwining operators in rank one versus higher rank. The technique there 
(essentially due to Gindikin-Karpelevic and later Schiffmann) is the origin of 
the change of variables used in Proposition 9.4 but here it did not lead to a 
simple product structure. 

Triple products. For a co-compact discrete subgroup T < G recall that 
Y = T\G, X = T\G/K and there is the Plancherel decomposition 

L\Y) - ©^gm^ 
and _ 

where G s denotes the subset of G which corresponds to the unitary spherical 
representations. 

As before we let (V'i)ie/ be an orthonormal basis of Maafi forms of L 2 (X), 
also considered as an orthogonal system in L 2 (Y). Note that i^iiTg) = 9 v i = 
{tt\. (g)vQ, rj) for some unitary principal series representation (tt\ ,Ti.\ ) and 

Let X e b 1 be an extreme point; if ip = 6 V0:V is a Maafi form, then 
■0e = ®(v )e,v G L 2 (Y) for all < e < 1. As before we have the identity 

iei 

for all < e < 1. Taking the norms, we have again 



(9.4) ii^n 2 = Ei^i 2 feii 2 = Ei^i 2 iiW) 



We now introduce new coordinates on by means of the simple roots; 
i.e., we will write A = A^ai + A' 2 a2- As norm on a£. we use the maximum norm 
||A|| =max{|A' 1 |,|A / 2 |}. 
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Theorem 9.11. Let G = S1(3,R) and T < G be a co-compact discrete 
subgroup. Then for every Maafi form ip corresponding to a unitary principal 
series there exists a constant C > such that for allT > 1, for the coefficients 
Ci of the Fourier series of tp 2 = X)j 6 /Cj^j, one has 

J2 IcilVM < CT 12 . 
l|Ai||<r 



Proof. Given the previous estimates, the pattern of proof follows that of 
Theorem 7.6. and is left to the reader. □ 

Remark 9.12. Friedberg, in [Fr87], determines precise gamma factors for 
the Rankin-Selberg convolution for T = Sl(3, Z). By Stirling's approximation 
it is easily seen that the exponential growth of these gamma factors differs 
from the exponential term in Theorem 9.11. A likely explanation for this is 
that the classical Rankin-Selberg integral is computed using Eisenstein series 
off a maximal parabolic subgroup. Our results have used Eisenstein series or 
Maafi forms associated to the minimal parabolic. We think that any relation- 
ship of the exponential factors must involve the embedding parameters of the 
representation into the principal series off the minimal parabolic. We thank 
the referee for bringing this point to our attention. 

Generalizations to Sl(n, R). Much of what has been said so far for Sl(3, R) 
can be generalized easily to G = Sl(n, R), n > 3. We will be interested in the 
radial limits of the spherical functions <p\, A G a£, in the imaginary direction 
of the first fundamental weight 

X UI = diag( , , . . . , ). 

n n n 

First we need the splitting formula for the spherical functions analogous to 
Proposition 9.4. The splitting will be accomplished for the choice of subalge- 
bras n' = QSE + Q~ a and n" = aes + Q~ a . Then n = n' © n" and we 

note that n" = R n_1 is abelian. The splitting formula in Proposition 9.4 then 
generalizes to 

(9.5) 

f n ~ l 1 

<£ A (exp(i7r(l - e)X UJl )) = C(\,e) / _ }J — dx x . . . dx n -i, 

JM n 1 j = i Jj{ £ i x ji ■ ■ ■ 7 x n—l) 

where 

fj{e, x,,..., x n _i) = \e^-^ +x] + ... + x 2 ^ (e~ 1 ^ +x 2 j + ... + x 2 n ^) 1 
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for all 1 < j < n — 1 and C(A, e) is bounded when e — ► for a fixed A. With 

(9.5) one easily concludes as in Lemma 9.5 that 

(9.6) v?A(exp(i7r(l - e)X U)1 )) < 

for all < e < 1 and a constant C only depending on A. Also, the proof of the 
lower estimate in Proposition 9.8 immediately generalizes to 

(9.7) v?A(exp(i7r(l - e)X ul )) > Ce K ^~ £ ^ sup ^ew a w.\(x u , 1 ) 

now for a constant C independent of A. Also, the proof of Proposition 9.9 
generalizes easily to G = Sl(n,R). For a co-compact subgroup T < G we get 
that 



(9i 



-(ra+ 



for all < e < 1. Clearly (9.5)-(9.8) also hold for ±irw.X u)l , w G Wo, instead of 
7rA Wl . Wc introduce now a norm on by setting ||A|| = sup wg yy Q \X(w.X UJ1 )\. 
Then the method in the proof of Theorem 9.11 together with (9.5)-(9.8) give 
the following estimates on triple products: 

Theorem 9.13. Let G = Sl(n, M) and T < G be a co-compact discrete 
subgroup. Then for every Maafi form ip corresponding to a unitary principal 
series the coefficients Ci of the Fourier series of tjj 2 = J2i£i c i^i there exists a 
constant C > such that for all T > 1, one has 



E 



< CT 



2rH 



-2 



Appendix A: The case G = Sl(2, R) 

In this appendix we deal with the group G = Sl(2, R) where we can per- 
form the most explicit computations. We think this is still of interest since it 
is the guiding example on which one can make conjectures for the general case. 
In particular, some of the ideas of the proofs in Section 1 are present in the 
explicit computations below. 

For G = Sl(2, R) note that G C G c = Sl(2, C) and G c is simply connected. 
We let t = so(2), 

a = {( X n M:*€R} and n = { ( ° " ) :n£ R}. 
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For z eC*, x eC and 9 G C we set 
and 

/ cos 9 sin# \ 

G if C - 



Note that 



— sin 9 cos 



4& = {a z : Re(z) > 0} and A* c = {a z : | arg(z)| < J}. 



Proposition A.l. Lei G = S1(2,R). Then the following assertions 

hold: 

(i) For all a z G and 9 G M, 

" 2 Av; G K c a z 'Nc 
with a' z G and 2/ defined by 



z' = \Jz 2 + sm 2 9{^- z 2 ). 

(ii) C iV c . 



Proof, (i) Set / = {z G C:|arg(z)| < §} and fix 9 G R. Let ft = 
{z G I:a z kQ G i^c^-c-^c}- By Lemma 1.4 the set ft is open and not empty. 
We have to show that ft = /. For z £ £2 define analytic functions z'(z), (p(z), 
x(z) such that 

a z k e = k^az'^n^y 

Writing this identity in matrix form yields 

/ z cos 9 z sin 9 \ _ / z' cos </? z'x cos 99 + \ 
^ _siM ^ )-{ -z'simp -z'xsm<p+ c ^ ) ' 

Thus we get 

z 2 cos 2 9 + = l- 2 ') 2 cos2 V + (^') 2 sin2 ¥ 

or equivalently 

(A.l) ( z ')2 = z 2 +sin 2£ ( J__ z2)> 

Taking real parts in (A.l) yields 

(A.2) Re(z') 2 = (1 - sin 2 9) Fie(z 2 ) + sin 2 9 Re(^) > 0. 

z l 
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We conclude that 




is a well-defined holomorphic map. 

Assume that SI / /. Then there exists a sequence (z n ) ne ^ in Q such that 
z = lim z n G 

Prom (A. 2) we now conclude that z' = lim^oo z'(z n ) exists in /. Now 
(A.l) implies that the limits ip = lim n ^ oc ip(z n ) and x = ]ini n ^ 00 x(z n ) exist. 
Thus 

a z k g = lim a Zn k e = lim k v < Zn) a z ,< Zn) n x < Zn) = k v a z m x G A c AcA c , 

contradicting a 2 /cg ifc-^c-^C- This proves (i). 

(ii) This follows from (i). □ 

Remark A. 2. (convexity theorems). For a linear semisimple Lie group G 
with Iwasawa decomposition G = KAN Kostant proved two convexity theo- 
rems (cf. [Kos73]): the linear convexity theorem which asserts 

(A.3) (VA G a) p a (Ad(K).X) = conv(W a .A) 

with pa'-p — ► a the orthogonal projection with respect to the Cartan-Killing 
form, and the nonlinear convexity theorem which can be stated as 

(A.4) (Va G A) a{aK) = exp ( conv(W a . log(a))) . 

For G = S1(2,R) a simple calculation shows that (A.3) extends to 

(VA G ac) Pa c (Ad(A).A) = conv(W a .A) 

with j»a c : Pc ~^ a c the complex linear extension of p a and we conjecture that 
(A.3) holds for all semisimple Lie groups G. Also it is natural to ask whether 
the nonlinear convexity theorem (A.4) generalizes to elements a G A^ (this 
makes sense in view of A\,K C K C A^-N C (cf. Proposition A.l(ii))). The 
answer is no and we can already see this for G = S1(2,K). Here we have 
W = ^2 = {1, s} with s.a = a' 1 for all a G A c . For z,w eC let 

h,w = {^z + (1 - X)w: < A < 1} 

be the line segment in C connecting z and w. Then Proposition A.l shows 
that for all z G C* with | arg(z)| < | that 

a(a z K) = {a w :w G (^2^-2)2}. 

Note that for z = e' 1 ^, < \ip\ < \ we have 1 {a^,:?/? G (^2^-2)2}, but 
1 G ex.p({a w :w G 1-%^,^}). Thus we see that (A.4) usually does not hold for 
elements a G Aq\A. 
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